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Preface 



This document was prepared as part of the MT-LAB research centre. The 
research centre studies the Modelling of Information Technology and is a 
VKR Centre of Excellence funded for five years by the VILLUM Foundation. 
You can read more about MT-LAB at its webpage www.MT-LAB.dk. 

The goal of the document is to serve as an introduction to new PhD stu- 
dents addressing the research goals of MT-LAB. As such it aims to provide 
an overview of a number of selected approaches to the modelling of stochas- 
tic systems. It should be readable not only by computers scientists with a 
background in formal methods but also by PhD students in stochastics that 
are interested in understanding the computer science approach to stochastic 
model checking. 

We have no intention of being encyclopedic in our treatment of the ap- 
proaches or the literature. Rather we have made the selection of material 
based on the competences of the groups involved in or closely affiliated to 
MT-LAB, so as to ease the task of the PhD students in navigating an other- 
wise vast amount of literature. 

We have decided to publish the document in case other young researchers 
may find it helpful. The list of authors reflect those that have at times played 
a significant role in the production of the document. 
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Chapter 1 
Introduction 



Model checking is a structured approach to system analysis. In this roadmap 
we put special emphasis on the analysis of transition systems with stochastic 
features. In particular we focus on systems that can be described by finite 
Markov chains in discrete or continuous time. In some sense the field has 
a history that goes back approximately 100 years, however, the structured 
approach offered by model checking is more recent and has had most of its 
formulation during the last 20 years. 

The roadmap is intended to give the newcomer to the fast in- 

troduction to the main ideas of the field serving as an introduction to the 
vast literature in the fields of stochastic model checking and performance 
evaluation. 

In Chapter [2] we give the definition of finite state Markov chains in dis- 
crete and continuous time. Markov chains can be used to describe (labelled) 
transition system. We discuss this in Chapter [3] Chapter [4] contains some 
important mathematical models useful for many model checking problems 
and relevant for performance evaluation in general. In Chapter [5] different 
languages used for model formulation. This is followed by chapter [6] describ- 
ing enquiry languages, i.e. logical expressions about the models that is to 
be tested. The actual testing is typical performed using concrete software 
packages. Some important examples are described in Chapter [7| Not all 
properties of interest can conveniently be expressed as logical requests. In 
this case one would term the analysis performance evaluation. This line of 
thought is pursued in Chapter [8} The distinction between model checking 
and performance evaluation is somewhat subtle but in general model check- 
ing is associated with the structured approach of logical requests to models 
formulated in a computer algebra. 
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Part II 



Fundamental Models in 
Stochastic Systems 
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Chapter 2 



Markov Chains: A 
Mathematician's Perspective 

A Markov chain is a model of a dynamical system with randomness where 
either the state space or time or both are discrete. We will focus on the 
discrete state-space. The most important property of a Markov chain is that 
the part of past behaviour with impact on the future of the process can be 
summarised in the current state of the process. 

We will let J(t) denote the state of the process at time t. If t is an element 
of the integers or another countable set then J(t) can be very general. How- 
ever, in most cases J(t) will also belong to a countable set, or less frequent 
to some Euclidean space. If t is an element of the reals then J(t) will belong 
to some countable set. 

2.1 Discrete Time and Discrete Space 

This is the basic case. The discrete state space can be finite or infinite. 

P(J(t)=j\J(t-l)=i)= Pij 

It is customary to collect these probabilities in a matrix P = {pij}- The 
parameters are called the one-stop transition probabilities. 

n (t) = 7r(t - 1)P = tt(0)P* 

It is customary to introduce the matrix P(t) = P*, the (i,j)th element 
of which has the probabilistic interpretation P{J{t) = j\J(0) = i). These 
probabilities are called the i-step transition probabilities. Suppose that there 
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is a path of non-zero probability for any state in the state space to any other 
state. In that case the limit 

1 * 

lim - > 77 (t) = 7T 

t^oo t ^ 
k=0 

exists. There are two possibilities, either tx is zero or all entries are positive. 

2.2 Continuous Time and Discrete Space 

The majority of continuous time Markov chain models used in computer 
science have a finite state space. A finite continuous time Markov chain can 
be viewed as a discrete time Markov chain where the time between state 
changes is exponentially distributed with a rate that depends on the current 
state. A continuous time Markov chain is parameterised by its generator 
matrix Q. A generator matrix is characterised by having all row sums being 
zero and by having non-negative off-diagonal elements. The absolute value 
of the diagonal element in row i gives the rate of the exponential distribution 
governing the time spent in state i. If state i is absorbing then this rate is 
0. The probability transition matrix P(t) is given as a matrix-exponential of 
the generator matrix 

P(t) = e Qt 

where e^' = J2T=o ^k\~- The marginal probabilities p(t) are given by 

p(t)=p(0)P(t). 

If there is a path of non-zero probability for any state in the state space to any 
other state then p(t) — > n, where 7r can be found as the only non-negative 
solution to 

7TQ = 

where the elements of tx sum to one. 

In computer science models one frequently encounter the possibility of 
events happening that leads back to the originating state. This possibility 
is not modelled with the generator matrix but can be dealt with in various 
ways. 



Chapter 3 



Markov Chains: A Computer 
Scientist's Perspective 



In general, dynamical systems fall into three categories. Discrete event sys- 
tems are characterised by a discrete but not necessarily finite state space 
and may, e.g., be described by state transition tables. In contrast, Continu- 
ous state systems are characterised by a continuous and immediately infinite 
state space and may, e.g., be described by differential equations. Finally, 
hybrid systems have the characteristics of both discrete event and continu- 
ous state systems. In this road-map document we shall be concerned only 
with discrete event systems, whereas continuous state and hybrid systems 
are covered by other MT-LAB road-map documents. 

A simple way of characterising discrete event systems is afforded by the 
notion of transition systems, i.e. pairs, T = (S, — >), consisting of a (possibly 
infinite) set of states, S, and a transition relation, — >Q S x S, defining the 
possible or allowed set of state changes. In the following we shall write s — > t 
to say that (s,t) G — >. 

Example 1 (Transition System) The transition system 
T = ({s 1 , s 2 , s 3 }, {(si, s 2 ), (s 2 , S3), (ss, si)}) 

abstractly characterises a system with three possible states and three transi- 
tions allowed between these. The system may be drawn as follows: 
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Mostly, however, our interest in systems goes beyond the possible states 
and transitions in an effort to understand various aspects of the processes that 
may take place in the system. In general, a process may be characterised as a 
sequence of changes to the state of an object. Without loss of generalisation, 
however, we shall always assume that the object of change is (a part of) the 
system itself. 

In order to fully accommodate our interest in processes we require the 
means to qualify the actual events that may cause, or be caused by, the 
system transitions. This requirement leads to the slightly more advanced 
notion of labelled transition systems. A labelled transition system (Its) is 
a triple, L = (S,Cab, — >), consisting of a set of states, S, a set of labels, 
£ G Cab such that S H Cab = 0, and a labelled transition relation, — >C 

S x Cab x S. Henceforth we shall write s — > t to mean that (s, £, t) G — >. 

Example 2 (Labelled Transition System) The labelled transition sys- 
tem 

L = ({si, s 2 , s 3 }, {a, b}, {(si, a, s 2 ), (s 2 , b, s 3 ), (s 3 , a, s 2 ), (s 2 , 6, si)}) 

abstractly describes a system with three states, two labels, and four distinct 
labelled transitions. The system may be drawn as follows: 

a b 

©oso® 

b a 

■ 

Note that we have carefully avoided giving a clear definition or explana- 
tion of the nature of labels. This omission has been made in order ensure 
the generality of our notion of labelled transition systems. Furthermore, the 
explained notions of transition systems do not define a start state. This is 
because the start state is sometimes immaterial to the function of the sys- 
tem and often immaterial to the system or process properties that interest 
us. Sometimes, however, a particular state will be singled out as the start 
state of an Its under consideration. We then say that the Its is rooted. 

3.1 Examples 

Clearly, the general notion of an Its may be instantiated in many ways. In 
the following we shall explain how to obtain several well known special cases 
from various branches of informatics. In the following we shall give some 
rudimentary such examples. 



3.1. EXAMPLES 
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3.1.1 Markovian Processes 
Markov Chains 

First we consider the situation where each event in the system is caused 
simply by a delay that expires. The underlying notion of time may, of course, 
be either discrete or continuous; as we shall see, either choice gives rise to a 
particular model. 

Let us first assume that time progresses in discrete steps. At each time 
step the system changes state in accordance with a transition probability 
vector, pi, that determines the probability, p^ G [0,1], of a change from 
the current state, Sj, to each state, Sj, of the system in a single time-step. 
Thus we know that \pi\ = \S\ for all i and we shall further demand that 
52fc=i Pij = 1 f° r a U * m order for the system to be well-formed. In this 
situation it is meaningful to choose the set of labels to be the set of permissible 
probabilities, i.e. p G [0,1], and the resulting notion of labelled transition 
system is exactly the class of Discrete Time Markov Chains (DTMCs). 

Example 3 (Discrete Time Markov Chain) 



Now let us assume that time progresses continuously. The length of each 
delay is an exponentially distributed stochastic variable, X G Ex(A), where 
the parameter, A G R, uniquely determines the underlying exponential dis- 
tribution. In this situation it is meaningful to choose the set of labels to be 
the set of permissible parameters, i.e. Cab = M, and the resulting notion of 
labelled transition systems is exactly the class of Continuous Time Markov 
Chains. 

3.1.2 Reactive Systems 

Note that, as in Example [4] below, the labels are often related to actions 
performed by system or user. Therefore the set of labels is sometimes referred 
to as the set of actions, a G Act. 

Example 4 (Event Driven State Machine) Consider the following la- 
belled transition system: 



1 



0.5 




0.5 



1 



Lamp = ({on, off}, {press}, {(off, press, on), (on, press, off)}), 
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It abstractly models a lamp. The lamp has two states, on and off, and it 
alternates between these state at the press of a button, which is the only event 
recognised by the system. The system may be drawn as follows: 



press 




press 



In Computer Science it is customary to think of systems as processes 
that navigate through a state space in discrete steps prescribed by a set of 
instructions. Such a system may be defined in any programming language 
(syntax), as long as the instructions of the language have a well-defined effect 
on the state of the system (semantics). 



Part III 
High-Level Modelling 
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Chapter 4 



Stochastic Modelling: A 
Mathematician's Perspective 



4.1 Notation 

In this section, mathematical symbols are involved to denote basic mathe- 
matical elements. A matrix is represented by capital primarily roman letters 
like T and A. The symbol / will be used for an identity matrix of appro- 
priate dimension, e is used to denote a column vector of ones of appropriate 
dimension, 



And denotes the vector of zeros. 



4.2 Phase- type Distribution 

The phase-type distribution is based on the method of stages, a technique 



introduced by A.K. Erlang 43 and generalised by Jensen [70] and M.F. 
Neuts [83]. The key idea is to model random time intervals as being made 
up of a (possibly random) number of geometric or exponential distributed 
segments and to exploit the resulting Markovian structure to simplify the 
analysis. The definition of phase-type distribution by M.F. Neuts [83] is: a 
probability distribution on the nonnegative integers is of phase type, if and 
only if there exists a finite Markov chain with a single absorbing state into 
which absorption is certain, such that for some choice of the initial probabil- 
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ities this distribution is that of the time till absorption. Continuous distri- 
bution on [0,oo) of phase type are similarly defined in relation to continuous 
parameter Markov chains. In general, many definitions and results regarding 
discrete time phase-type distribution carry over verbatim to the continuous 
time case, others need minor modifications. 



4.2.1 Discrete Phase-type Distribution 

A discrete phase-type distribution in a finite discrete time Markov chain with 
transition matrix P of dimension m + 1 is given by (4.1 ). The Markov chain 
has m transient and 1 absorbing state. 



(4.1) 



P = 

[0 1 

where T° = (I - T)e. 

The initial probability vector is denoted by (a, a TO +i). The pair (ex., T) is 
called a representation of the phase-type distribution. Given a representation 
of a discrete phase-type distribution, we can calculate the probability mass 
function by 

f(x) = ctT x ~ 1 T°, x > 0. (4.2) 
Thus, the cumulative distribution function is given by 

F(x) = 1 - QT x e, x > 0. (4.3) 

The probability generating function of a discrete random variable is a 
power series representation of the probability mass function of the random 
variable. The generating function H(z) from a non-negative discrete random 
variable X is given by 

00 

H(z) = E(z x ) = Y,z X f(x)> (4-4) 

x=0 

where f(x) is probability mass function. For a discrete phase-type random 
variable, we find 

00 00 
H(z) = = «m+i + ^2W- 1 T° = a m+1 + za(I - zT^T . 

x=0 x=l 

(4.5) 

From the generating function H(z) we can obtain the factorial moments 
for a discrete random variable by successive differentiation. For a discrete 
phase-type variable with representation (a, T) we get the factorial moments 

E(X(X - 1) . . . (X - (Jfe - 1))) = k\cx(I - T)- k e, k > 1. (4.6) 



4.2. PHASE-TYPE DISTRIBUTION 
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The matrix U = (I — T) _1 is of special importance as the (i,j)th element 
has an important probabilistic interpretation as the expected time spent in 
state j before absorption conditioned on starting in state i. 



4.2.2 Continuous Phase-type Distribution 

A continuous phase-type distribution in a finite continuous time Markov 
chain with infinitesimal generator matrix Q of dimension m + 1 is given 



by (4.7). The continuous time Markov chain has m transient and 1 absorbing 



state. 



Q 







(4.7) 



where T° = (I - T)e. 

The initial probability vector is denoted by (a, a m +i). The pair (a, T) is 
called a representation of the phase-type distribution. Given a representation 
of a continuous phase-type distribution, we can calculate the probability 
density function by 

f(x) = ae Tx T°, x>0. (4.8) 
Thus, the cumulative distribution function is given by 

F(x) = 1 - ae Tx e, x>0. (4.9) 

Let U= (— T) -1 , then the (i,j)ih element Uy is the expected time spent 
in state j given initiation in state i prior to absorption. Thus, we have the 
first moment of continuous phase-type distribution PH(ck,T) as aUe, the 
mean of a distributed random variable. 

To calculate either probability density function or cumulative distribution 
function involves matrix exponential computation (i.e. e Tt ). There is a 
very efficient method for the calculation of this matrix-exponential called 
uniformization . Introducing the quantity 9 = -min(Tjj: 1 < % < m) one 
rewrites T=6(K-I). The matrix K is a sub-stochastic matrix such that K 
= I +r 1 T. Now 

(9t) l K l 



e Tt = e- et 



Formula (4.10 ) is very well suited for numerical evaluation as all terms in the 
series are non-negative and since an appropriate level for truncation of the 
sum can be derived from the Poisson distribution. 
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4.2.3 Closure Properties of Phase-type Distribution 

One of the appealing features of either discrete or continuous phase-type dis- 
tribution is that the class is closed under a number of operations. The closure 
properties are a main contributing factor to the popularity of phase-type dis- 
tributions in probabilistic modelling of technical systems. In particular the 
class is closed under addition, finite mixtures, and finite order statistics. 



Sum of two independent PH variables 



In both discrete and continuous cases, consider two random variables X 
and Y with representation (ol,T) and (J3,S) respectively. Here the dimension 
of T is m and the dimension of S is k. Then the random variable Z = X+Y 



follows a phase-type distribution with representation (7, L) given by (4.11 ). 



T T°$ 
S 



(4.11) 



and 7 = (a, a m+1 f3), % 



m+k+l 



a 



Finite mixtures of phase-type distributions 



In both discrete and continuous cases, given X{ phase-type distributed 
with representation (oti, Tj) we have Z = hXi with h = 1 and P(Ij 
= 1) = Pi). It is easy to see that the random variable Z is itself phase- type 
distributed with representation (7, L) given by (4.12). 



Ti ... 
T 2 ... 

... T k 



(4.12) 



and 7 = {p 1 oc 1 ,p 2 cx2,. . . ,p k Oi k )- 



Finite order statistics 



The order statistic of a finite number of independent discrete phase-type 
distributed variables is itself phase-type distributed. Given X phase-type 
distributed with (T x , a x ) and Y phase-type distributed with (T y , cx y ) is 
min(X,Y) phase-type distributed with representation (7, L) given by 



L = T X ®T, 



(4.13) 



4.2. PHASE-TYPE DISTRIBUTION 
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where 7 = a x (g> a y . Further max(X,Y) is phase-type distributed with rep- 
resentation (7, L) given by ( 4.14[ ). 



T x ®T y T X ®T° T x <g> Ty 
















(4.14) 



and 7 = (a x ® ayja^a^m+^a^it+iaj,). Here the dimension of T x is k and 
the dimension of T y is m. We write L° explicitly: 



T-lO 

a; 
T-iO 

y 



In the continuous case, for X G PH(T X , a x ) and Y G PH(T y , ck^) 
min(X,Y) is phase distributed with representation (£,7) given by (4.15): 



L = T X ® I y + I X ®T X 



v 



(4.15) 



where 7 = cx x £g> a y . And max(XY) is phase type distributed with repre- 
sentation (£,7) given by : 



T x ®Iy + I x ®Ty 4®T y T£®Iy 








T 








■a 



(4.16) 



and 7 = (cKa;(8)Q; y , a x a y)m +i, oc Xj k+iOc y ), where the dimension of T x is k and 
the dimension of T y is m. We give -L explicitly 





rpd 
X 
rpO 

L V. 



4.2 .4 Non-uniqueness of representations 

A main drawback when modelling with phase-type distributions is the non- 
uniqueness of their representations. In other words, a phase-type distribution 
is uniquely given by any representation. However, several representations can 
lead to the same phase-type distribution, similar to the bisimulation relation 
of state transition systems. The different representations of a phase-type 
distribution constitute an equivalence relation. 



28 



CHAPTER 4. MODELLING (MATHEMATICS) 



4.3 Queueing Theory 



Queueing theory 49 57] involves the mathematical study of queues, or wait- 
ing lines. The formation of waiting lines is a common phenomenon that oc- 
curs whenever the current demand for a service exceeds the current capacity 
to provide that service. The ultimate goal of study is to achieve an economic 
balance between the cost of service and the cost associated with waiting for 
that service. In real life, queueing systems are surprisingly prevalent in a wide 
variety of contexts. Four broad classes of queueing systems are commercial 
service systems, transportation service systems, business-industrial internal 
service systems, and social service systems. Therefore, queueing theory con- 
tributes on many decisions in reality by predicting various characteristics of 
the waiting line such as the average waiting time. 



4.3.1 Basic Structure of Queueing Models 



Queueing system 



Input 


Customers : 




Queue 


■ ► 


Service 


Served 


source 






mechanism 


customers 



Figure 4.1: Basic structure of queueing models 



The basic structure of queueing progress is depicted in Fig. 4.1 Cus- 



tomers requiring service are generated over time by an input source. These 
customers enter the queueing system and join a (finite or infinite) queue. At 
certain times, a member of the queue is selected for service by some rule 
known as the queue discipline (e.g. first-come-first-served). The required 
service is then performed for the customer by their service mechanism, after 
which the customer leaves the queueing system. 



4.3.2 Kendall's notation 



In queueing theory, models conventionally are labelled by Kendall's notation 
for characterising, depicted in fig 4.2 

The most common process is denoted as 



4.3. QUEUEING THEORY 
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Distribution of service Number of places in the 

times system 

- I - I - I - 

Distribution of interarrival 

Number of servers 

times 



Figure 4.2: Kendall's notation 



M = exponential distribution (Markovian) 

D = degenerate distribution (constant times) 

Ek = Erlang distribution (shape parameter = k) 

G = general distribution (any arbitrary distribution allowed) 



4.3.3 Terminology and Notation 

The state of a queueing model is typically defined as the number of cus- 
tomers in the system. In standard terminology, N(t) denotes the number 
of customers in queueing system at time t (0 < t). The system begins in 
an initial state, and the state of system will change as time elapses. The 
system is said to be in a transient condition. However, after sufficient time 
has elapsed, the state of the system becomes essentially independent of the 
initial state and the elapsed time. The system has now essentially reached a 
steady-state condition. Queueing theory has tended to focus largely on the 
steady-state condition, partially because the transient case is more difficult 
analytically. 

By eliminating the number of customers being served from the state of 
queueing model, the queue length can be obtained. Some standard quantities 
considered in queueing theory are 

s = number of servers (parallel service channels) in queueing system 

A n = mean service rate of new customers when n customers are in system. 
In case A n remains constant, this constant is denoted by A. 

fi n = mean service rate system when there are n customers in the system. 
Note : n n represents combined rate at which all busy servers achieve 
service completions. When the mean service rate per busy server is 
constant, this constant is denoted by \x (In this case, ji n = s/i). 
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p = A/(s/i) is the utilisation factor for the service facility, where it rep- 
resents the fraction of the system's service capacity (sp) that is being 
utilised on the average by arriving customers (A). Note : p < 1 is the 
usual stability criterion in queueing models, where the queue attains 
an equilibrium as t — > oo. There is no steady-state condition if p > 1. 

Steady-state condition 

L = expected number of customers in queueing system. 

L q = expected queue length (excludes customers being served). 

W = E(cj), where u = waiting time in system (includes service time) for 
each individual customer. 

W q = E(u q ), where uj q = waiting time in system (excludes service time) for 
each individual customer. 

The relationships between L, L q , W and W q are given as Little's formula. 

1. L = XW 

2. L q = XW q 

3. W = W q + I 

Note : Little 's formula is extremely important in a sense that they enable all 
four of the fundamental quantities to be immediately determined as soon as 
one is found analytically. 



4.3.4 Types of Queueing Models 

There have been many studies on various types of queueing models. The 
systems, in order M/M/l, M/G/l, G/M/l and G/G/l, give the insights 
from the special queues with Markovian characteristics to greater generality, 
which is also roughly in order of increasing difficulty We refer to chapter 11 
in 



49 and chapter 15 in 57 for further reading. 



4.4 Renewal Process 

A renewal process is a recurrent-event process with independent and iden- 
tically distributed inter-event times. It is a generalisation of the Poisson 
process, where the exponential assumption from the Poisson process is re- 
laxed. The asymptotic behaviour of a renewal process is described by the 
renewal theorem and the elementary renewal theorem. 



4.4. RENEWAL PROCESS 
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4.4.1 Definition 



The general distribution of renewal process is depicted in fig 4.3 



A 



N(t) 



VX 1+ 



x 4 



X, 



-* — *- 



Figure 4.3: A renewal process 



N(t): The number of occurrences of some event in the time interval [0,t] 
Xi : zth inter-arrival time 
Tk : time of the kth arrival 

The formal definition of a renewal process N = {N(t) : t > 0} is a process 
such that 

N(t) = max{n : T n < t}, 

where T = 0, T n = X\ + X 2 H h X n for n > 1, and {Xj} is a sequence of 

independent identically distributed positive random variables. 

In renewal process, the first interval is allowed to have a different dis- 
tribution. Let Xi be distributed according to a distribution H(t) and X^ 
(k > 2) be distributed according to a distribution F(t). If H(t) = F(t), the 
process is called a renewal or more standard ordinary renewal process. If 
H(t) ^ F{t), the process is called a modified or delayed renewal process. 
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4.4.2 Theorems 

In renewal theory, study focuses on distribution of N(t) and moments of 
N(t). A fundamental relationship in renewal process is 

P{N(t) <n} = P{T n > t}. 

The expected number of events, E(N(t)), is given by renewal function, de- 
noted by m(t). Let Fk be the distribution function of Tjt, renewal function is 
calculated by 

oo 

m(f) = = 

n=l 

The method of Laplace- Stieltjes transforms (see Definition of Appendix 
I in 1 49 1) is often useful in renewal theory. The renewal function is expressed 
as a Laplace-Stieltjes transform 

m*(e) = H *^ 9 } n , for 6^0. 
v ; 1-F*(0) r 

For H(t) = F(t), standard ordinary renewal process 

m *($) = F *^]^ for 6^0. 
K J 1 - F*(0) 

For large values of t, let /x = E(Xi) (i > 2) be the mean of a typical 
inter-arrival time, then we have the elementary renewal theorem. 

-mit) — > — as t — y oo. 
t w /x 



We refer to chapter 10 in 49 for further reading. 



Chapter 5 

Stochastic Modelling: A 
Computer Scientist's 
Perspective 



5.1 Performance Evaluation Process Algebra 
(PEPA) 

The Performance Evaluation Process Algebra, commonly known as PEPA, 
was invented by Jane Hillston during her PhD studies, 1991 - 1994, at the 
University of Edinburgh. The resulting dissertation, published as part of 
the Distinguished Dissertations in Computer Science series (59), remains the 
authoritative reference to the calculus. At the time PEPA was not alone but 
emerged as one of a number of stochastic process algebras (SPAs) developed 
roughly at the same time |20(|28||48). 

A key motivation for the development of PEPA, and indeed the use of 
process algebra based paradigms for performance modelling, was composi- 



tionality 58 . A compositional approach allows complicated systems to be 
modelled in a systematic manner. Simple low-level components are modelled 
first and higher-level components are subsequently modelled as compositions 



of low-level ones. Thus, as in pure process algebra 19,64,80 , a system is 
modelled as a complex of interacting components. The behaviour of each 
component is defined by the actions that it can perform or as a composition 
of smaller components. In contrast to pure process algebra, though, actions 
are not assumed instantaneous. Instead, each action is associated with an 
exponentially distributed random variable that characterises its duration. 

The early SPAs all agreed on a core set of algebraic combinators: The 
action prefix, (a,r).P, describes a system that is ready to perform action 
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a at the rate r and then go on to behave as described by P. The choice, 
P + Q, denotes a process that may go on to behave either as described by P 
or as described by Q. The action hiding construct, P/L, restricts the scope 
of visibility for the actions mentioned in the set L to cover only P. Finally, a 
notion of parallel composition that is based on the CSP concurrency operator 
64 , P\aQ (written P\\lQ in non-PEPA SPAs), denotes a system composed of 

L 

wo processes, P and Q, that may go on to unfold their individual behaviour 
independent of one another except for actions mentioned by the cooperation 
set, L, on which the two processes must synchronise. Thus the syntax of 
the early SPAs is generally given by (a variant of) the following grammar for 
PEPA: 

P ::= (a,r).P | P + Q | PwQ | P/L | A 

Formally, the behaviour modelled by a PEPA term is a labelled transition 
system inductively defined by the rules and axioms of the following structural 
operational semantics: 



P^Q . 



clef 



(a,r).P^4p A ^4q 



i£A = P 



p^4p' q { ^4q' 



p + q ( ^Ip' p + q^4q' 

pHp 1 q^4q' 

7^7, if a L 7^7, if a £ L 

PmQW P'wQ PmQ^4 PmQ' 

L L L L 

p^ pi p faj pi 

-. — r if a 4. L -, — r if a G L 

( a ' r ) t-,1 I t r> I t ( r ' r ) 



P/L ^ P'/L P/L ^ P'/L 

P ( ^P> Q^Q' ifaGL 

PWQ^ P'W Q' aIld Wllere R = rI(P) rI(Q) mill ( r « ( P ) ' r " (Q) ) 

Note, that the first eight of these rules are practically shared by all of the 
early SPAs. The last rule, however, expresses Hillston's view on the nature 



of synchronisation as encoded in PEPA 58 . Each of the early proposals 
represented a particular view on this aspect that is formally expressed in 
the definition of the duration of the delay that occurs when two actions are 
synchronised. This issue has proven to be a both crucial and controversial 
aspect of design in stochastic process algebras. 
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In PEPA the perception of synchronisation is governed by both concep- 
tual and pragmatic concerns. Conceptually PEPA takes the view that action 
synchronisation represents the cooperation of two equal partners; hence the 
total time of the event is determined by the slower of the two partners [59]. 
Pragmatically PEPA insists that rather than just a labelled transition sys- 
tem the behaviour of a model must be a discrete time Markov chain. The 
latter of these concerns dictates that the resulting duration must be charac- 
terised by an exponentially distributed random variable. The former concern 
requires that the rate, R, that characterises the distribution must be related 
to the minimum of the rates, r\ and T2, that characterises the delays of the 
individual partners. 

These concerns motivated Hillston to define the duration of the resulting 
delay as an exponentially distributed random variable characterised by the 
rate, R = - r ^ - %r min(r a (P), r a (Q)). Here r a (P) denotes the so-called 
apparent rate of the action a in the sub-process P, i.e. the sum of the rates of 
all occurrences of a in P that might compete to participate in the interaction. 
So - denotes the probability that the action (a,ri) is the participant 
from P, r v ?q^ is the similar probability of (a,r2) being the participant from 
Q, and mxa(r a (P),r a (Q)) selects the lowest apparent rate as a base rate, 
thereby effectively expressing the idea that the slowest participant decides 
the duration. Note, that situations where one participant is active and the 
other passive is easily modelled in this framework by allowing oo as a rate 
for some actions. 



Over the years Hillston's choices have been subject to some debate. On 
the one hand, the main point of the critics is that the class of exponential 
distributions is not closed under maximum; hence PEPA's characterisation 
of the duration of synchronisation cannot be accurate in all situations and 
must be considered a pragmatic approximation. On the other hand, the main 
argument of the proponents is that the computational advantages of Markov 
chains over more general representations outweighs the imprecision of the 
approximation. 



Regardless of the debate, Hillston's ideas have arguably been very influ- 
ential and a number of other process algebras have adopted her notion of 
synchronisation 86 , 87 . The PEPA calculus has also been very successful 



from a practical point of view and a number of tools now support PEPA [24J 
[32j|46j|63j|94] . In terms of usage PEPA has by now proven its value in a num- 
ber of contexts, such as traditional performance modellin g [40|[47|[62|[65] , 



network security 25 27 , 97 98 , and systems biology 29,30,44]. 
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5.2 Interactive Markov Chains (IMC) 

Interactive Markov Chains (IMC) was developed by Holger Hermann's dur- 
ing the 1990'es. The resulting algebraic formalism is perhaps best described 



by 26 . As the language is very expressive the standard notion of Markov 
Chains is not a sufficient semantic model. For this reason Hermann's Phd 
dissertation, published as a monograph in the series of Springer Lecture 
Notes [55), takes the position that the Interactive Markov Chain is itself 
a fundamental (semantic) model, related to Markov Decision Processes and 



Markov Automata |41|67 71 , accompanied by an algebraic syntax. In the 
following we shall assume the view of [26] and present IMC as a process 
algebraic formalism that requires a rich semantic model. 

The fundamental goal of IMC is to provide a compositional methodology 
for modelling and analysis with Markov Chains |55|. Like other SPAs the 
language relies on traditional process algebraic notions and primitives in 
order to achieve this goal. Thus, we assume a countably infinite set of action 
labels, a G A, denoting visible actions as well as a single distinguished internal 
action, r, and write a G A T = AU {r} to denote actions in general. Further, 
we assume a countably infinite set of process variables, X G V. In this context 
the following syntax describes the (sequential) process algebraic fragment of 
IMC: 

V 3 P ::= I a.P | P + Q \ X \ [X := P] J ... 

As customary for process algebraic languages, 0, is used to denote the 
terminal process that is stuck and can perform no further actions. 

In contrast, the action prefix, a.P, describes a process that is ready to 
perform action a and then go on to behave as described by P. 

The choice, P + Q, denotes a process that may go on to behave either as 
described by P or as described by Q. 

Finally, the shorthand notation [X := P] is used for an arbitrary (finite) 
set of defining equations of the form 

X x := P x 
v ■ P 

■ 1 n 

with Xj G V, and Pj complying to the above grammar. These equations 
denote a set of mutually recursive processes, where each Xj denotes an entry 
point that is essentially just a named internal state. The subscript i is used 
to denote the currently active such internal state (or equation). 

It is fundamentally assumed that actions take no time, hence the expres- 
sions of this fragment simply denote labelled transition systems in accordance 
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with the following structural operational semantics: 

P i {[X:=P] i /X s }-±>P' 

a.P^P [X:=P].^P' 

P^P' Q^Q' 
P + Q^P' P + Q^Q' 

In order to facilitate modelling with Markov chains the syntax is enriched 
with the delay prefix, (A).P, that describes a process that is ready to delay 
for an amount of time that is exponentially distributed with parameter A G R 
and then go on to behave as P once the delay expires. 

The additional expressiveness offered by this extension is obvious from 
the fact that any Markov chain can be modelled by the following syntactic 
fragment: 

V 3 P ::= | (A).P | P + Q | X | [X := P]. | . . . 

The correspondence between such algebraic expressions and Markov chains 
is captured by the following structural operational semantics: 

P i {[X:=P] i /X j }-Up' 

W.p-Up [x^p^-Up' 

P -U P' Q -U Q' 

p + q-Up' P + Q -U Q' 

As already pointed out, any action labelled transition system, often called 
a reactive or interactive system, can be modelled in the process algebraic 
fragment of this language. As is usually the case for such systems is 
simply an ordinary binary relation for any distinct a because a.P + a.P ~ 
a. P. Similarly, any Markov chain can be described using the Markovian 
fragment. In this case, however, an ordinary relation does not suffice because 
(A).P + (A).P w (2A).P. In order to accommodate this we have to accept 

A 

that > is a binary multi-relation for any distinct A. 

When the full language is used, however, actions and delays can be in- 
terleaved in arbitrary manners. Thus the resulting models are neither reac- 
tive/interactive systems or Markov chains. Instead we obtain an interactive 
Markov chain - a model as expressive as the continuous time Markov de- 
cision process but notationally more convenient. In particular, the use of 
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action labels allows the sought after compositionality to be realised using 
the two final language primitives: 



V 3 P ::= hide a± • • • a k in P \ P \ • • • a k ] \ Q | ... 

The parallel composition, P \ [a\ • • • a k ] | Q, denotes a system composed of 
two processes, P and Q, that may go on to unfold their individual behaviour 
independent of one another except for actions mentioned by the interaction 
set, ai ■ ■ • a k , on which the two processes must synchronise. 

The action hiding construct, hide a\ • • • a k in P, restricts the scope of vis- 
ibility for the actions mentioned in a± • • • a k to cover only P. This is used to 
control the synchronisation structure. 

Semantically, the composition of two IMCs gives rise to a new IMC in 
accordance with the following rules of structural operational semantics: 



P P' 



a £ a 1 ■ ■ ■ a k 



a f. ai • • • a k 



P\[ ai ---a k ]\Q ^ P'\[ ai -.-a k ]\Q 

Q^Q' 

P\[a 1 ---a k ]\Q^P\[a 1 ---a k }\Q' 

Q^Q> P^P' 
P\[a 1 ...a k ]\Q^P>\[a 1 ...a k ]\Q> a ^ a i' ' ' a k 
P - > P' 

^ ■ p a ■ ■ ■ . T^T a £ «1 • • • a k 

hide a 1 ■ • • a k in P — > hide a t ■ ■ ■ a k in P 

P - > P' 

f — a G ai • • • a k 

hide ai • • • a k in P — > hide ai ■ ■ ■ a k in P 

P -U P' Qr^ 

P\[a 1 ---a k ]\Q -U P'\[a 1 ---a k ]\Q 

Q -U Q' P-^ 
P\[a 1 ---a k ]\Q -U P\[a 1 ---a k \\Q' 

A T 

P > P' hide ai • • • a k in P 

A 

hide a± ■ ■ ■ a k in P > hide a± • • • a k in P' 



Like PEPA the language of Interactive Markov Chains is very much in- 
fluenced by the multi-way synchronisation paradigm of CSP. Hence, the first 
five of the above rules encode this synchronisation paradigm and are more 
or less standard in the CSP school of process algebra. In adopting these 
rules IMC further encodes the unique view on delays that separates it from 
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other SPAs, e.g. PEPA: Actions are instantaneous and NOT associated with 
delays. 

The remaining three rules encode the view that unhindered synchroni- 
sation (i.e. t which is instantaneous) always takes precedence over delays. 
This is known as the maximal progress assumption. Note, that the hiding 
operator serves to delimit the scope of synchronisations, thereby allowing us 
to distinguish between potentially blocked, i.e. a, and definitely unhindered 
interactions, i.e. r. 

Two consequences of these rules are particularly noteworthy: 

First of all, IMC, in contrast to PEPA, does not define any way for delays 
to synchronise in a single exponential delay. Instead, IMC insists that delays 
interleave, thereby giving rise to compound delays the duration of which is 
phase-type distributed. 

Second of all, non-determinism cannot always be completely resolved in a 
closed IMC. For this reason neither the action labelled transition system nor 
the Markov chain suffice as semantic models. This is the reason why IMC 
itself is also cast semantic model. 

In general, proponents of IMC are very critical towards PEPA. As previ- 
ously mentioned, the main point of criticism is that the class of exponential 
distributions is not closed under maximum. Indeed a central claim of IMC is 
that the phase type distribution arising from the interleaving of delays is the 
natural (and correct) characterisation of the maximum of the two involved 
exponential distributions. The price to pay for this increased correctness, 
however, is a significantly larger state space induced by the embedded phase- 
types as well as the aforementioned non-determinism, which forces us to use 
a more complicated semantic model. 

Other stochastic process algebras include: Timed Processes for Perfor- 
mance evaluation (TIPP) [48], Markovian Process Algebra (MPA) |20|, Stochas- 
tic 7r-calculus (87] , Extended Markovian Process Algebra (EMPA) |21|, BioAm- 



bients [89], MoDeST [22], and StoKlaim (a stochastic extension of Klaim) |84|. 

Other stochastic modelling formalisms include: Stochastic Automata Net- 
works (SAN) [85] and Stochastic Petri Nets (SPN) [81]. 
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Part IV 
Model Checking 
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Chapter 6 



Logical Specification of 
Stochastic Properties 

To verify a property of a system (regardless the type of system), we first need 
some way of expressing it. While we can always describe specific properties 
in a suitably rich mathematical logic — e.g. first order logic — we need to 
limit this expressiveness if we are to automatically verify any property that 
we can write down. More specifically, the challenge is to find a logic that is 
both expressive enough for the properties we are interested in, and admits 
efficient model checking algorithms. 

For the qualitative analysis of concurrent systems, a number of temporal 
logics were developed with this aim. In practice the two most widely used are 
Computation Tree Logic (CTL) [42] and Linear Temporal Logic (LTL) [79] , 
which are both subsets of the logic CTL*. The difference between CTL 
and LTL is in the treatment of time — an LTL formula refers to a specific 
execution path in the model, whereas a CTL formula refers to a tree of 
possible computations. 

For a model with N states and M transitions, and a property $, the 
complexity of CTL model checking is 0((N + M)|$|), whereas LTL model 
checking is 0((N + M)2'*'). On the other hand, it is generally considered 
to be more intuitive to specify properties in LTL [95]. As an example, con- 
sider the following formulae constructed from the 'next' (X) and 'future' (J 7 ) 
modalities (we will introduce these properly later in the chapter): 

• (LTL) VXJ 7 ®: on all paths, after the current state there is a state in 
the future at which $ holds. 

• (LTL) VJ-X $: on all paths, there is a state in the future for which the 
next state satisfies $. 
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• (CTL) \tX VJ r< l ) : on all paths out of all next states, there is a state in 
the future for which $ holds. 

• (CTL) VJ 7 \/X $: on all paths, there is a state in the future for which 
all successor states satisfy $. 

The first three of the above have identical semantics, however the last formula 
means something quite different — illustrating the care that must be taken 
when writing CTL formulae. 

In the context of qualitative properties — relating to probability, time, 
and rewards — there have been a number of logics developed that extend 
CTL. The fact branching time was favoured over linear time is most likely 
because of the lower complexity of the model checking problem. The seman- 
tics and model checking algorithms of both CTL and LTL can naturally be 
extended to a probabilistic setting. Since there are an infinite number of 
computation paths in any non-trivial model, we need to assign probabilities 
to measurable sets of paths. In CTL, computation trees naturally form mea- 
surable sets of paths (so-called cylinder sets), and since either the satisfaction 
or violation of a given LTL formula can be demonstrated by a finite prefix 
of a path, we have a similar construction in both cases |13] . 

Both explicit state 63 , 72 and symbolic [63] model checking algorithms 
have been extended to probabilistic systems. In the former, the graph reach- 
ability algorithms are essentially modified to solve probabilistic reachability 
problems. In the latter, Binary Decision Diagrams (BDDs) are extended to 
Multi- Terminal Binary Decision Diagrams (MTBDDs), which are efficient 
data structures for representing real-valued functions (i.e. those that map to 
a probability), rather than just Boolean functions. 

In this chapter, we will describe the following logics, which are the ones 
most widely used in practice: 





Discrete Time 


Continuous Time 


Without Rewards 


PCTL (Section 


6.1 


) 


CSL (Section 


6.3 


) 


With Rewards 


PRCTL (Section 


6.2 


) 


CSRL (Section 


6.4 


) 



Although all of the above are branching time logics, the logic PCTL* [13] — a 
probabilistic extension of CTL* — has also been introduced, and is supported 
by the PRISM model checker [63]. As with CTL*, the complexity of PCTL* 



is exponential in the size of the formula. Note that it is doubly exponential, 
however, when there is non-determinism in the model. There has been some 
analogous work in the context of CSL, using regular expressions to describe 



path formulae 14 15 



Before we describe each of the above logics in detail, let us introduce 
the basic syntax that is common to each. Importantly, there is a distinction 
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between state formulae and path formulae, in that the former hold of states in 
the model, whereas the latter hold of (possibly infinite) sequences of states, 
or paths. We will use $ to denote state formulae, and tp to denote path 
formulae. 

All the above logics contain the following state formulae in common, 
where a G AP is an atomic proposition, or label of a state: 

$ ::= tt | a | $ A$ | | V< p ((p) 

This consists of a propositional fragment, along with a probability measure 
formula. The latter states that the probability measure over paths that 
satisfy ip is <j p, where < G {<, <, >, >}. 
The common path formulae are as follows: 

if ::= X<& | $W$ 

X $ is the untimed next operator, which holds of a path if a if the next state 
o~\ satisfies $. $1 U $2 is the untimed until operator, which holds of a path a 
if some state in the future satisfies $ 2 , and all states before this point satisfy 
$1: Eli Oi |= $ 2 A Vj < i Oj |= $1. 

Note that various commonly-used operators can be derived as follows: 

T§ = ttW$ 

g<& = n(ttWn$) 

$iW$ 2 = (6$i)V($iW$ 2 ) 

J 7 $ is the future operator, which states that $ holds at some point in the 
future along a path. Q $ is the global operator, which states that $ holds 
for all states along a path. $ x is the weak until operator, which states 

that $1 has to hold along the path until $ 2 holds (but $ 2 might not ever 
hold). Finally, $i7?.$2 is the release operator, which states that $ 2 must 
hold up to and including the point at which $1 becomes true, and if $1 never 
becomes true, then $ 2 must hold forever. 

We will describe the semantics of this common subset of the logics (and 
subsequently, for each logic we introduce), in terms of DTMCs and CTMCs. 
A DTMC is a three-tuple (S,P,L), where S is a non-empty finite set of 
states, P : S x S — > [0, 1] is a stochastic matrix, and L : SxAP — > { tt, f f } is 
a labelling function. A CTMC is a four-tuple (S, P, r, L), where r : S — > M>o 
gives the exit rate of each state, and S, P, and L have the same meaning as 
for a DTMC. 

A path a in a DTMC is a (possibly infinite) sequence of states so, s±, . . . G 
S, such that for all % < \a\ — 1, P(s i: s i+ i) > 0. We write a[i\ = s i: 
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and Paths(s) to be the set of paths such that o~[0] = s. A path a in 
a CTMC is a (possibly infinite) alternating sequence of states and dura- 
tions s , t , Si,ti, . . ., such that Sj G S", tj G M>o, and for all i < |cr| — 1, 
P(si, Si + i) > and r(sj) > 0. As for a DTMC path, we write cr[i] = Sj, but 
we additionally define 5(a, i) = U (the time spent in state Sj), and a@t = cr[i], 
where i is the smallest index such that t < X}j=o^r 

The semantics of the common subset of the logics is as follows, s \= $ 
means that a state s satisfies a state formula $, and a \= <p means that a 
path cr satisfies a path formula (p. Note that the semantics of the untimed 
next and until path formulae are the same for paths in both DTMCs and 
CTMCs, since there is no reference to time. 
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tt 


iff 


true 
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iff 


L(s, a) 
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$1 A $ 2 
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s = $i and s = $ 2 
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^$ 


iff 
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h 




iff 


Pr{ cr G Paths(s) cr |= y9 } < p 


a 


h 




iff 


cr[l] |= $ 


a 


h 




iff 


3z. = $2 and Vj < i. cr[j] |= 



We will now describe each logic in turn. We will extend the above seman- 
tics in each case, but just describe the syntax, and the sort of properties that 



can be expressed. In Section 6J3 we will conclude with a brief discussion of 
uniformisation. In the next chapter, we will then relate these logics to the 
property specification languages of the PRISM and MRMC model checkers. 



6.1 PCTL 

Probabilistic Computation Tree Logic (PCTL) (53] is a logic for specifying 
properties of DTMCs and MDPs, and has the following syntax: 

$ ::= tt | a | $A$ | -i$ | V< p (ip) \ 

Here, we have introduced a time-bounded until operator. $iZY- n $ 2 states 
that $2 will hold at some time t < n, and until that time, $i will always 
hold. When n = oo, this is the same as the unbounded until operator. More 
formally, the semantics is: 

a |= $! U- n $ 2 iff < n. a[i] (= $ 2 and Vj < i. a[j] |= $i 
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In the MRMC model checker, the bounded until operator has been extended 
to take an arbitrary interval / = [ti, t 2 ], such that t 1 G N> , t 2 G N> U {oo}, 
and ti < t 2 . This has the semantics: 

a |= $iW 7 $ 2 iff 3i G I. a[i] \= $ 2 and Vj < i a[j] \= $i 

Explicit state model checking of PCTL can be performed by recursively 
checking the sub-terms in a formula. The only interesting case is the until 
operator, for which we need to calculate the probability of the formula hold- 
ing in each state. For both the bounded and unbounded until operator, we 
first label the states that definitely satisfy the property (<3>2 holds), or def- 
initely fail to satisfy the property (neither $1 nor $ 2 holds). The bounded 
until operator $1 U- n $ 2 then corresponds naively to raising to probability 
transition matrix of the DTMC to the power n, followed by a matrix-vector 
multiplication. We can improve the efficiency, however, using dynamic pro- 
gramming. The unbounded until operator corresponds to solving a set of 
linear equations. 

6.1.1 Long-run properties in PCTL 

In both PRISM and MRMC, PCTL has further been extended to allow long- 
run properties. More specifically, they support a long-run operator £< p ($), 
which states that the probability of satisfying $ in the steady state is <j p. 
Mathematically, a steady state distribution is only defined for an ergodic 
DTMC — specifically, one that is positive recurrent (for every state in the 
DTMC, the expected period before returning to it is finite) and aperiodic (it 
is not the case that any state can only return to itself in multiples of k > 2 
time steps). The semantics of the long-run operator is slightly more general 
though (requiring only aperiodicity): 

s |= £<„($) iff lim Pr{ a G Paths(s) \ a[n] |= $ } < p 

n—tco 

In practice, this means that the model checking algorithm consists of two 
stages. First, we identify the bottom strongly connected components (BSCCs) 
and compute the probability of reaching each BSCC from every transient 
state. We then compute the steady state distribution of each BSCC, which 
by definition must be ergodic (we require aperiodicity, and a BSCC is by 
definition irreducible, which implies positive recurrence when there are only 
finitely many states). 
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6.1.2 PCTL* 

We can extend PCTL by allowing path formulae to be nested — in other 
words, allowing us to specify arbitrary LTL path formulae. This leads to the 
logic PCTL* (13): 

$ ::= tt | a | $A$ | -i$ | V< p (<f) 

(f ::= $ | Xf | plA p | pU- n p | p A f \ —up 

Other than the nesting of path formulae, the only difference to PCTL in the 
above syntax is that we allow conjunction and negation of path formulae. 
The semantics of path formula is as follows, where we write a % to be the 
suffix a[i], cr[i + 1], . . . of the path cr: 

a \= $ iff a[0] |= $ 

a \= X if) iff cr 1 |= if 

a \= f\U.f2 iff 3i. <j % \= p 2 and Vj < i. cr- 7 |= f\ 

a |= f i U- n p 2 iff 3? < n. a % |= p 2 and Vj < i. a- 7 |= f\ 

a \= fx A fi iff cr |= fx and a |= f>2 

a \= -if iff cr Y 1 ¥ 

PCTL* model checking is performed in a bottom-up recursive manner, 
in the same way as for PCTL. The difference is that when we reach a path 
probability operator, we convert the path formula into a Quantitative LTL 
Specification (QLS). This is a pair, (if, I), consisting of an LTL formula ip 
(including the bounded until operator), and a probability interval of the form 
[0,p] or [p,l], for p E [0,1]. 

Verification of a QLS formula amounts to constructing an w-automaton 
corresponding to the path formula, and taking the product of this with the 
original system (a DTMC or MDP). This is often a Rabin automaton^) since 
we require the w-automaton to be deterministic — deterministic Biichi au- 
tomata do not accept all w-regular languages, whereas deterministic Rabin 
automata do. Alternatively, deterministic Streettj^jor Mullei|^] automata are 
sometimes used. The QLS model checking problem is then reduced to prob- 
abilistic reachability (essentially, PCTL model checking) on the product au- 
tomaton. 

lr The acceptance condition of a Rabin automaton is a set of pairs (Ei, Fi) — a string 
is accepted if it results in a sequence of states where for some i, there is at least one state 
in Fi that is visited infinitely often, and all states in Ei are visited finitely often. 

2 The acceptance condition of a Streett automaton is the negation of the Rabin condi- 
tion. 

3 The acceptance condition of a Muller automaton is a set of sets of states F — a string 
is accepted if the set of states it visits infinitely often is an element of F. Rabin and Streett 
automata can be described as Muller automata. 
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Note that the complexity of PCTL* model checking is doubly exponential 
in the size of the formula — this can be reduced to singly exponential in the 
case of a DTMC, but not in the case of an MDP. PCTL* is supported by 
PRISM for both DTMCs and MDPs, but not by MRMC. 

6.2 PRCTL 

PCTL is used to specify properties of a DTMC, but we might want to build 
a reward structure on top of such a model. Rewards (or equivalently, costs) 
can be assigned to either states or transitions, or both. These can be used to 
capture a variety of metrics, including power consumption, monetary cost, 
and quality of service. To reason about such reward structures, Probabilistic 
Reward Computation Tree Logic (PRCTL) [fl] was developed as an exten- 
sion of PCTL. The syntax is as follows: 

$ ::= tt | a | $A$ | -1$ | V< p (cp) | £< p ($) 
if ::= 

This is PCTL, extended with four additional state formulae, that allow us to 
reason about reward rates (£), instantaneous rewards (C), and accumulated 
rewards (y). The only change to the path formulae is the addition of a 
reward bound on the time-bounded until operator. $i Uj $2 means that $2 
holds within a number of steps n 6 I, that all states before this satisfy $1, 
and that the accumulated reward before satisfying $2 is in the interval J. 
Note that we use a superscript to talk about time, and a subscript to talk 
about rewards. 

In the original paper [IT] , the only path formula considered is the time- 
and reward-bounded until operator, hence we only show this in the above. 
The other PCTL path formulae can also be included, however, and are sup- 
ported by MRMC. 

The reward operators are: 

• £ j(3>) — the long-run expected reward per time unit, in states that 
satisfy is within the interval J. 

• £j{&) — the expected reward per time unit, in the first n time steps, 
in states that satisfy $, is within the interval J. 

• Cj($) — the instantaneous reward at the nth time step, in states that 
satisfy is within the interval J. 
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• y](&) — the expected accumulated reward until the nth transition, in 
states that satisfy $, is within the interval J. 

We interpret the semantics of PRCTL over a Discrete Markov Reward Model 
(DMRM). This is a 4-tuple (S,P,L,p), where (S,P,L) is a DTMC, and 
p : S —> lR>o is a reward structure, which assigns a real value to each state 
that represents the reward accumulated when we leave the state (note that 
a self loop is interpreted as leaving and then re-entering the state). We can 
describe the PRCTL semantics more formally as follows: 



s 


1= 




iff 


lim Pr{ a G Paths (s) a[n] = $ } < p 


s 






iff 


g( S ,{s'\s'\=^})eJ 


s 






iff 


g(s,{s' | s' \=$},n) e J 


s 




cm 


iff 


p(s, {s' s' \= G J 


s 






iff 


y(s,{s' \s' = $},n) G J 


a 


1= 




iff 


3i G /. a[i] \= $2 and Vi < j. a[i] = $ 



and J^jJo P( a [j}) e J 



where we make use of the following reward measures: 

1 n 

g(s,S',n) = — E (p(°s\i})) 

i=0,a s \i]eS' 

g(s,S') = lim g(s,S',n) 

n— >oo 

p(s,S',n) = ^ p(s')n(s,s',n) 
s'es> 

n-l 

y(s,S',n) = ^2p(s,S',i) 

7r(s,s',n) = 7r(s, t, i) • ir(t, s', n — i) for < i < n 

tes 

Here, ir(s,s',n) denotes the probability of being in state s' after n steps, 
given that we start in state s, and its definition comes directly from the 
Chapman-Kolmogorov equations. 

The PRCTL reward operators are supported directly in MRMC, whereas 
PRISM has a slightly different syntax for reward-based properties. We will 
discuss this in the next chapter. 

6.3 CSL 



Both PCTL and PRCTL are discrete-time logics, used in the context of 
DTMCs and MDPs. We can move to continuous time with only minor 
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changes in the logics. The biggest difference comes in the model checking al- 
gorithms themselves, in that uniformisation-based techniques are employed. 



Continuous Stochastic Logic (CSL) [12,16 is a logic for expressing properties 



of CTMCs, and has the following syntax: 

$ ::= tt | a | $A$ | -i$ | V< p {<p) | <S« P ($) 
if ::= X§ | | $W 7 $ 

The main difference between PCTL and CSL is that since the latter is a 
continuous time logic, the intervals on the path operators can be real- valued. 
The operator S is a steady state operator, and is just the continuous time 
analogue of the long-run operator C of PRCTL. 

As an example of the sort of properties we can express, consider the 
following CSL formula, for AP = { Error, Completed }: 

V> . 9 {^ErrorU [0 ' w] Completed) 

This will be satisfied by all states from which there is a probability of at 
least 0.9 that we will reach a 'Completed' state within 10 time units, without 
encountering any 'Error 1 states before that point. Note that the unit of time 
is implicit to the model, and is only relevant with respect to our interpretation 
of the results. 

More formally, the semantics of the new CSL operators are as follows: 
s \= <W$) iff lim Pr{ a E Paths (s) | a@t \= $ } < p 

t— >oo 

a \= ® 1 U I $ 2 iff 3t e I. a@t (= $ 2 and Vtf < t. a@t' (= $x 
In addition to the above operators, as presented in |16j, we can additionally 



define a time-bounded next operator 18 . The path formula X 1 $ requires 



that the next state satisfies $, and that the transition will take place in 
the time interval /. Such an operator does not make sense in the context 
of PCTL, where transitions occur at discrete points in time (hence their 
duration is abstract). The semantics of the timed next operator is as follows: 

(7h^$iff |= $ and S(a, 0) e I 

The standard model checking algorithm for the CSL time-bounded until 
operator is based upon uniformisation — we have to consider the three cases 
of [0, t], [t, oo], and £2] for the time interval, but the algorithm is essentially 
a first passage time analysis [l7j . For the timed next operator, the model 
checking algorithm boils down to a matrix-vector multiplication. Note that 
the untimed next and until operators can be model checked as per PCTL by 
considering the embedded DTMC. 



52 



CHAPTER 6. STOCHASTIC LOGICS 



One problem we face with CTMCs is that uniformisation does not pre- 
serve the validity of all CSL formulae. This is a problem if we perform 
lumpability based abstractions, since these are typically based on a uni- 



formised CTMC 73 . We will write CSL\X to mean the subset of CSL 
without the next operator. If two CTMCs are weakly bisimilar, then the 
validity of all CSL\X formulae is preserved fl8l. A consequence is that the 



uniformisation of a CTMC preserves CSL\X equivalence. 



6.3.1 CSL Variants 



Path-based reward variables are regular expressions (expressed as finite state 
automata) that allow us to reason about more complex behaviours than with 
the standard CSL path operators [68]. This idea has subsequently been 
explored in a logical fashion, by allowing regular expressions in place of path 
formulae. 



in 



The first such variant of CSL that was proposed was pathCSL 14 
which path formulae are time-bounded regular expressions, concerning se- 
quences of pairs of state formulae and action types. This assumes that the 
transitions on the CTMC are labelled with an action type a G Act, for some 
finite set Act: 



$ ::= tt | a | $A$ | | P<pO) | <S< P ($) 

<p :: = (ap$ 

a ::= e I $a I aa I a + a I a* 



Here, a is a regular expression over the alphabet of pairs of state formulae 
and action types — S = { $a | $ is a state formula, a G Act }. 

Path formulae are interpreted over finite paths a = so,a Q , . . . , s n , a n _i, s n , 
which consist of alternating states G S and actions a, G Act. As previously, 
a@t is the state of a at time t, and 5(a, i) is the residence time in state Sj. 
We write cr(£i, t 2 ) to mean the fragment of the path between times ti and t 2 
inclusive, and c(a) to be the completion time for the path a (i.e. the total 
time taken for the path). The semantics of the path formula (a;)-*$ is as 
follows: 



a |= («}-*$ iff 3t' G [0,*]. a@t' h * and cr(0,t') G Paths(a) 



Here, the set Paths (a) of all paths that satisfy the regular expression a is 
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defined recursively as follows: 



a 


e 


Paths(e) 


iff 


cr = s G S 


a 


e 


Paths($a) 


iff 


a = so, a, si and so |= $ 


a 


g 


Paths{ot\a 2 ) 


iff 


3t' G [0,c(a)}. a(0,t') G Paths^) 










and cr(t',c(cr)) G Paths(a 2 ) 


a 


e 


Paths(ai + a 2 ) 


iff 


cr G Paths (ai) or cr G Paths (a 2 ) 


a 


e 


Pathsi a*) 


iff 


3i G N. cr G Pathsia 1 ) 



pathCSL was later extended to asCSL (CSL with actions and state la- 
bels) [l5], which is essentially the same logic, but with minor syntactic 
changes. Again, transitions on the CTMC are labelled with an action type, 
but properties can refer to actions types a G Act U {\/}, where y/ G" Act. 
The pseudo-action y/ is always immediately executable and does not change 
the state of the CTMC — its main use is at the end of a regular expression, 
allowing us to write ($, ^/) to require the final state to satisfy $, without 
requiring any particular action type afterwards (or indeed, any subsequent 
transition at all). 

The syntax of asCSL is as follows: 

$ ::= tt I a I $A$ I -1$ I P< P (^) | «S<p($) 
if ::= a 1 

a ::= e | ($, a) | a; a | a U a \ a* 
The semantics of path formulae is as follows: 

cr |= a 1 iff there exists a finite prefix a' of cr such that 

cr G Paths (a) and c(cr) G / 



Paths(a) is defined recursively as before (we denote by cxj the prefix of a up 
to state and by a 1 the suffix of a from state Sj onwards): 



a 


G 


Paths(e) 


iff 


\<t\ = 


a 


G 


Paths{§ } a) 


iff 


a = s , a, si and s \= $ and 5(a, 0) > 


a 


G 


Paths{$,s/) 


iff 


a = s and s \= $ 


a 


G 


Paths (ai, a 2 ) 


iff 


3i < cr . (Tj G Paths (ai) and cr 1 G Paths (a 2 


a 


G 


Paths(a>\ + a 2 ) 


iff 


a G Paths(ai) U Paths(a 2 ) 


a 


G 


Pathsi a*) 


iff 


Bi G N. a G Pathsia 1 ) 



These ideas have not yet been implemented in the PRISM or MRMC model 
checkers, however the use of regular expressions for transient analysis has 
been developed and implemented in the context of PEPA [31]. Here, they 
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are called stochastic probes — a regular expression is written, that specifies 
a sequence of actions to observe in the PEPA model. The question we then 
ask is: what is the first passage time until we reach a state where we have 
observed a sequence of actions that is in the language described by this 
regular expression? A common example of such a property is in measuring 
the response time of a component in the model. 

A stochastic probe is compiled into a PEPA component that corresponds 
to the regular expression, in much the same way as the automata-theoretic 
approach to LTL model checking. Importantly, the component only contains 
passive activities, and so it does not alter the behaviour of the original model 
(it only adds additional information by introducing additional states). The 
problem then reduces to a first passage time analysis. This is supported by 



the PEPA plug-in for Eclipse 94 



Note that both pathCSL and asCSL are in a sense less expressive than a 
hypothetical CSL*, in which the CSL path formulae can be nested. This is 
because, in the above, a single time bound is given for the entire path. Hence 
model checking can be performed via an automata product construction, sim- 
ilar to LTL model checking, followed by a time-bounded reachability analysis, 
or first passage time analysi^J As an example of the difference, consider a 
path formula X^ ' 1 ^ X^ ' 1 ^ $, which states that we perform two transitions, 
each within one time unit, to reach a state satisfying $. This is clearly dif- 
ferent to (X X $)[°' 2 ], expressible in asCSL, which states that we perform 
two transitions within two time units, to reach a state satisfying $. The first 
formula is stricter, since a smaller set of paths will satisfy it. 



6.4 CSRL 

In the same way as with discrete time models, we can add reward structures 
to CTMCs. The main difference is that the reward assigned to a state is not 
a fixed reward that is given for each time step we occupy that state, but a 
rate of reward acquisition. 

To reason about reward-structured CTMCs, the Continuous Stochastic 
Reward Logic (CSRL) 1 33 1 was developed as an extension of CSL. It has the 
following syntax: 

$ ::= tt | a | $A$ | | V< p ((f) | <S< P ($) 

<f ::= x}$ | 



4 Note that the computation of a first passage time and a time to absorption is the 
same, if we make the target state absorbing. 



6.5. A NOTE ON UNIFORMISATION 



55 



The only extension to CSL is the addition of the time- and reward-bounded 
next and until operators. Xj $ states that the next state satisfies and 
the transition is made at some time t G I, and the accumulated reward until 
time t is in the interval J. The until operator $2 is the continuous-time 

analogue of the corresponding PRCTL formula. 

The semantics of CSRL is interpreted over a Continuous time Markov 
Reward Model (CMRM). This is a tuple (S,P,r, L, p, l), where (S,P,r,L) 
is a CTMC, p : S — > M>o is a reward structure describing the rate per time 
unit at which a reward is accumulated in each state, and t:Sx54 lR>o 
is a reward structure describing the impulse reward accumulated when a 
transition is made between two states. It must be the case that for all s G S, 
i(s, s) = 0. We define the accumulated reward along a path a at time t to 
be: 

(i-l \ i-l i-l 

j=0 / i=Q i=0 

The semantics of the new operators in CSRL is then as follows: 
s \= S<p($) iff lim Pr{ a e Paths(s) | a@t (= $ } < p 

t— >oo 

a (= AfJ $ iff tr[l] |= $ and 0) G / and y a (5(a, 0)) G J 
a \= $ 1 U I j $ 2 iff 3t G J. a@t |= $ 2 and Vt' < *. a@f |= $ x 

and 2/o-(t) G J 

There is no analogue in CSRL of the PRCTL S, C, and y state formulae, 
which means that we cannot reason about the long-run expected rate of a 
reward, the expected reward up to time t, the instantaneous rate of reward 
at time t, or the expected accumulated reward until time t. We can still 
express accumulated rewards over individual paths, however, thanks to the 
reward-bounded path operators. MRMC supports CSRL directly, whereas 
PRISM uses its own reward operators, which are the same for both continu- 
ous and discrete time properties — they allow us to talk about reachability, 
cumulative, instantaneous, and steady state rewards (6|. We will describe 
these in detail in the next chapter. 

6.5 A Note on Uniformisation 

The heart of the model checking algorithms for computing time-bounded 
probabilistic reachability properties in CTMCs are based on uniformisation. 
Recall that we previously presented a CTMC as a four-tuple (S, P,r, L), 
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where S is the state space, P describes the transition probability between 
each pair of states, r describes the exit rate of each state, and L is a labelling 
function. If we assign a numerical index to each state s G S, then we can 
write P as a stochastic matrix — i.e. the rows all sum to one. 

An alternative way of characterising a CTMC is in terms of an infinites- 
imal generator matrix Q. If we consider a CTMC with N states, such that 
P is its probabilistic transition matrix (an N x N stochastic matrix), and 
r(i) = r(s) describes the exit rate of state s with index < i < N, then we 
can define the elements of Q as follows: 

Q(i,j) = r(i)P(i,j) ifi^j 

(6.1) 



Q(i,i) = -r(i)^P( 



If r(i) is a constant rate A for all i, then the CTMC is said to be uniformised, 
and we can write Q in the following form: 

Q = X(P - I) 



In general, this is not the case, but given Q as defined in Equation |6\T 
and a unformisation constant A, we can construct a uniformised probability 
transition matrix P as follows: 

— Q 
P = ^ + l 

A 

We require that A > X m in = maxj \Q(i, i)\ in order for P to be a stochastic 
matrix. 

For transient analysis, as used by the model checking algorithms to com- 
pute probabilistic reachability properties (a first passage time analysis), it 
is best to choose the smallest value of A possible. However, we need to be 
careful if we want to ensure that the embedded DTMC P of the uniformised 
CTMC preserved the ergodicity of Q. If we are not careful, it is possible for 
P to be cyclic, such that it does not converge on a steady state distribution 
for all initial distributions over the states. The PRISM model checker sets 
the value of A to be slightly greater than \ m in- 

It is interesting to note, however, that under different circumstances - 
depending on what the purpose of the unformisation is — a different choice 
of A may be optimal. One example of this is in the construction of stochastic 
bounds, where we try to find a DTMC whose steady state solution is an 



upper bound on that of the original DTMC, using a stochastic ordering 93 



The usual idea is to find an upper bound that exhibits some property such as 
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lumpability 75 , so that its state space can be reduced in size. Since stochastic 
bounds are formulated in terms of DTMCs, we need to apply them to the 
embedded DTMC of a uniformised CTMC — hence we need to choose an 
appropriate unformisation constant A. 
It has been shown in 



37 



that the optimal choice of A in this circum- 
stance is 2X min . Intuitively, this introduces additional probability mass to the 
diagonal of P, which gives more flexibility when constructing a stochastic 
bound, allowing the bound to be tighter. 
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Chapter 7 

Stochastic Model Checking 



In this chapter we will look at the capabilities of two of the leading prob- 
abilistic model checkers. We look at PRISM in Section 17.11 and MRMC in 
Section 17.21 



7.1 PRISM (version 3.3.1) 

The probabilistic symbolic model checker PRISM [7,78 was developed at the 
University of Birmingham by Marta Kwiatkowska's group and is currently 
being maintained and developed by the same research group at Oxford Uni- 
versity. The core team includes Marta Kwiatkowska, Gethin Norman and 
Dave Parker, and they have been developing PRISM since 1999. PRISM 
provides support for DTMCs, CTMCs, and MDPs, using a probabilistic 
guarded-command language (known as the PRISM language) as its primary 
interfacd^l 



An overview of the structure of the PRISM tool, taken from 77 , is shown 



in Figure 7.1 The model checking algorithms are based around three engines: 



• The sparse engine is based around a sparse matrix representations of 
the model, and corresponds to explicit state model checking. 

• The MTBDD engine is based on using multi-terminal binary decision 
diagram^] to represent the model, and corresponds to probabilistic 
symbolic model checking (this is where the name PRISM came from). 

1 PRISM also supports PEPA as an input language, but only active-passive synchroni- 
sation is implemented. A PEPA model maps onto a CTMC. 

2 An MTBDD is like a BDD, except that the terminal nodes represent probabilities (in 
this context), rather than truth values. If there are only a few probability values that an 
expression can evaluate to, this can lead to an efficient representation. 
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Figure 7.1: Overview of the structure of PRISM 



MTBDDs can be used to efficiently represent very large models that 
have a regular structure, but this is not the case for all models, and it 
depends on the specific ordering of variables in the PRISM model - 
in other words, the order in which variables are declared in the PRISM 
file can affect the size of the MTBDD representation of its transition 
matrix. 



The hybrid engine uses a combination of the above two representa- 
tions 76 . The transition matrix of the model is stored in an MTBDD, 
whereas the iteration vector — recording the probability of each state 
satisfying a given property — is stored as a full array. This gives better 
results than the MTBDD engine for most models, since the majority 
of states have different probabilities of a satisfying a given property, 
meaning that the iteration vector cannot be stored efficiently as an 
MTBDD. Because of this, the hybrid engine is selected in PRISM by 
default. 



We will give an overview of the PRISM language in Section 7.1.1 which 
maps onto a DTMC, CTMC, or MDP, and may be enriched with reward 
structures. We will then describe the PRISM property specification language 



in Section 7.1.2 The following logics are supported, for each type of model: 





PCTL 


PCTL* 


CSL 


Rewards 


DTMC 


/ 


/ 




/ 


MDP 


/ 


/ 




/t 


CTMC 






/ 


/ 



^Only certain reward properties (reachability and instantaneous rewards) are 
supported by MDPs. 
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Note that PRISM does not support PRCTL and CSRL as described in 
the previous chapter. Instead, it has its own syntax for reward specification, 



which we will describe in Section 7.1.2 PRISM does support CTL model 



checking in the context of MDPs, where the CTL property Aip has the same 
semantics as the PCTL property V>x((f). This is because <p must hold over 
all adversaries (even unfair ones) in order for the probability to lie in the 
interval [1, 1], and therefore be > 1. These two properties also have the the 
same semantics for a DTMC, since Zeno paths have probability zero. Hence 
the only sensible interpretation of A<p on a DTMC is that ip almost certainly 
holds. 



7.1.1 The PRISM Language 

The PRISM modelling language |6| is a simple, state-based language based on 
the Reactive Modules formalism |10|. It has two main components: modules 
and variables. A PRISM model consists of a number of modules that run in 
parallel. Each contains local variables that constitute its state — which may 
only take on a fixed range of values — and a set of guarded commands that 
describe its behaviour. Global variables are also allowed, which can be read 
and modified by all module^] 

As an example, consider a PRISM module with only two local variables, 
x and y, declared as follows: 

x : [0. .3] init 0; 
y : bool init false; 

x is an integer variable, taking a value in the set {0,1,2,3}, and y is a 
Boolean variable. There are eight possible states of the module (not all may 
be reachable), and we specify a state by giving the values of all the variables 
in the module. The initial state is given by the initial values of the variables, 
as specified in the declaration — in this case, (x = 0, y = false). We can 
specify a set of states in a compact way, by giving an inequality on the 
values of certain variables. For example, the condition x > 2 describes all 
the states where x has a value of 2 or 3, and y has any value (there are four 
such states). 

The behaviour of a PRISM module is described by a set of guarded com- 
mands. A guarded command has the following general form, and is prefixed 
with an optional action name A: 

[A]G^ Pl :U 1 + ---+p n :U n 

3 Commands that synchronise are not allowed to modify global variables, to prevent 
race conditions. 
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The guard G is a condition on the state of the variables (i.e. it determines 
a set of states in which the command can execute) - - both local to the 
module and in other modules. If G is true, then with probability Pi, update 
Ui is performed. It must hold that YliPi = 1j an d the probabilities must 
be specified — if the sum contains only one element, the probability can be 
omitted, and will be taken to be one. An update specifies how the state of 
the local variables changes (we write x to refer to the old state of a variable 
and x' to refer to the new state). If we are specifying a continuous time 
model, rates will be used in place of probabilities. 

In a DTMC or CTMC model, there must only be at most one guard that 
evaluates to true for each state of the module. In an MDP model, multiple 
guards can evaluate to true, which introduces local non-determinism. 

We build a PRISM model, known as a system, by specifying how to 
compose its modules. The following operators are supported, based on those 



in the Communicating Sequential Processes (CSP) process algebra 64 



Mi||M2 — parallel composition of modules Mi and M2, synchronising 
on all action names that appear in both Mi and Mi. 

M1IHM2 — asynchronous parallel composition of modules M\ and M 2 . 

Mj|[L]|M2 — parallel composition of modules M\ and M 2 , synchronis- 
ing only on action names in L. 



• M/L — hiding^] of action names in L, in module M. The module be- 
haves as M, except that action names in L are renamed to the internal 
action name r. 

• M/p — renaming of action name a in module M to p(a). 

When we compose two modules, any commands prefixed by an action name 
that they synchronise over must execute together. For discrete time mod- 
els, we take the joint probability distribution over the updates to perform 
after executing the two commands. In continuous time models, we similarly 
multiply each pair of rates together. 

PRISM uses an interleaved model of concurrency. This means that when 
two modules run in parallel, and can perform commands independently, there 
is a question as to the order of their interleaving. In an MDP model, this is 
treated as a non-deterministic choice, whereas for a DTMC model, the choice 
must be probabilistic. In PRISM, if their are n commands concurrently 



4 Note that this is hiding in the style of CSP, as opposed to restriction in the style of 
CCS. The latter blocks communication on names appearing in L. 
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PRISM modules 
in parallel 




DTMC composition 

Figure 7.2: Composition of PRISM modules 



CTMC composition 
(omitting loops) 



enabled, then the probability of executing each command is -. Intuitively, 
we can think of a probabilistic choice being made as to which command to 
execute, followed by the probabilistic choice specified in the command. This 



is illustrated in Figure 7.2 where the PRISM modules are shown graphically, 
and the enabled command on each has only one possible choice. 

For a CTMC model, the commands take place at a particular rate, rather 
than with a certain probability. When PRISM takes the composition of 
the modules, every enabled command in a state of the system is allowed to 
proceed at the specified rate. Intuitively, this means that the exit rate from 
a state of the system is the sum of the exit rates of the individual states, 
and the probability of one command proceeding over another depends on the 



relative exit rate. This is illustrated again in Figure 7.2, where the labels 
on the transitions should be interpreted as rates rather than probabilities. 
We have omitted the self loops, as they do not change the behaviour of the 
CTMC. 

The final feature of PRISM that we will talk about here is its reward 
structures. Each model can be given multiple reward structures, and each 
reward structure is defined separately to the rest of the model (i.e. outside 
any module definitions). A reward structure is simply a list of pairs of states 
and real values, corresponding to the reward (or cost) for that state. In 
practice, we specify such a structure compactly, by describing a set of states, 
and giving an expression for the reward in each case, in terms of the state 
variables. A simple example of a reward structure that only depends on one 
variable (x : [0 . . 10] ) is: 



rewards 
x < 5 : 
x >= 5 : 

endrewards 



20 * x; 
100; 
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This assigns a reward of 20x to all states in which x has a value less than 
5, and a reward of 100 to all the other states (when the value of x is between 
5 and 10 inclusive). 

7.1.2 Property Specification 

Properties in PRISM are based on the two main logics we described in the 
previous section — PCTL for discrete time models, and CSL for continuous 
time models. The atomic properties are just sets of states, which can be 
defined by constraints on the variables in the model, in the same way as 
when specifying reward structures. We can also define labels, which can 
subsequently be used as a shorthand for such sets of states. 

There are three operators that PRISM provides for state formulae — P for 
the probability of satisfying a path formula, S for the long-run or steady state 
probability of being in a certain set of states, and R for reward properties. 
The P and S operators are much the same as in PCTL and CSL, and have 
the following syntax, where $ is a state formula and (p is a path formula: 

P <r [ ip ] 
P =? [ (p ] 
S <r [ $ ] 
S =? [ $ ] 

The only difference is the addition of the qualitative properties, P =? and 
S =?, which evaluate to a probability. This has no impact on the model 
checking algorithms, since we have to compute the probabilities in any case, 
but is useful from a practical standpoint. In the case of an MDP, we need 
to replace '=?' with one of 'min=?' or 'max=?', depending on whether we 
want the minimum or maximum probability of the property holding. PRISM 
supports all the path formulae of PCTL, PCTL*, and CSL, including the 
derived CTL operators for convenience. 

Reward properties in PRISM are a little different to those in PRCTL and 
CSRL that we saw in the previous chapter: 

R <jr [ Rp r op ] 

R — ? I- Rprop -I 

R { reward structure } <r [ R pr0 p ] 
R { reward structure } =? [ R prop ] 

The information in braces is required for models with multiple reward struc- 
tures — this can either be the name of the reward structure, or an index 
(with 1 being the first reward structure, etc.). For the purposes of experi- 
mentation, an undefined integer constant can used in place of a fixed index. 
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As before, in the case of an MDP, we replace '=?' by 'min=?' or 'max=?', 
depending on whether we are interested in the minimum or maximum value 
of the reward. 

There are four types of reward property supported by PRISM: 

• F $ - - a 'reachability reward'. This specifies the expected reward 
accumulated along a path until a state satisfying $ is reached. If we 
allow quantitative properties in PRCTL (i.e. of the form V=?(<p), then 
we can express the property R <= r [ F $ ] as follows: 

7> =7 (ttZV]$) 
V=?(ttU&) 

• C <= t — a 'cumulative reward'. This specifies the expected reward 
accumulated along a path until a time t (which is either a natural 
number or a real number, depending on whether we are verifying a 
discrete- or continuous-time model). R <= r [ C <= t ] is equivalent 
to the PRCTL property yf r] (tt). 

• I = t — an 'instantaneous reward'. This specifies the expected reward 
(or rate of reward in the case of a continuous-time model) of the model 
at a particular time t. R <= r [ I = t ] is equivalent to the PRCTL 
property Cf 0>r] (tt). 

• S — a 'steady state reward'. This specifies the reward per time unit 
in the long run. R <= r [ S ] is equivalent to the PRCTL property 

£[0,r]("tt). 

All of the above are supported for DTMCs and CTMCs, but only reachability 
rewards and instantaneous rewards are supported for MDPs, and only by the 
sparse and MTBDD engines (not the hybrid engine). 

The property 'R <= r [ F $ ] ' is quite different from the PRCTL path 
formula tt 3>- The former is a state formula, requiring the expectation 

(over all paths) of the accumulated reward until $ holds to be < r. The 
latter, on the contrary, is a path formula, requiring the accumulated reward 
on a specific path to be < r. This means that in PRCTL we can impose 
a reward bound on every path that satisfies a certain path formula, but we 
cannot impose a bound on the expected reward over all paths that satisfy 
the formula. The opposite situation is true of PRISM reward properties. 

Note that in PRCTL, the reward operators are qualified with a state 
formula $ — for example, Cj (<£>) for instantaneous rewards. This means 
that we only include any contributions to the reward from those states that 
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satisfy <£>, when we verify the property. There is no equivalent to this in 
PRISM, but if we know in advance which states satisfy $, then we could 
manually encode this as a separate reward structure that only assigns a non- 
zero reward to these states. 

All of the above properties we have discussed are state formulae — that 
is to say, the model checking problem only makes sense if we say which state 
we are interested in. When we ask PRISM to verify a property, it makes the 
following assumptions: 

• If the property is Boolean, e.g. 'P < 0.01 [ F "error"] ', then PRISM 
will return true iff the property holds of all states in the model. If we 
are only interested in a particular set of states, we can use a logical 
implication. For example: 

"state_x" => P < 0.01 [ F "error"] 

• If the property is quantitative, e.g. 'P =? [ F "error"] ', then PRISM 
will return the value of the property for the initial state of the model. If 
we are interested in a particular state, we can use a filter, by identifying 
the state inside the quantitative operator. For example: 

P =? [ F "error" {"state_x"}] 

If we specify a set of states in this way, we also need to say whether 
we are interested in the minimum or maximum value of the property 
in this set. For example: 

P =? [ F "error" {"state_x_or_y"} {max} ] 

Note that there is a difference between the above '{max}' notation for 
the maximum value of a property over a set of states, and the 'max=?' 
notation for the maximum value of a property over all schedulers. For 
example, in the context of an MDP, we could write the following prop- 
erty: 

P max=? [ F "error" {"state_x_or_y"} {min} ] 

This allows us to talk about the smallest probability of there being an 
error in the future, starting from a certain set of states ("state_x_or_y"), 
given a worst-case adversary. 
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7.1.3 Simulation in PRISM 

In addition to stochastic model checking, PRISM supports discrete event 
simulation of models. There are two interfaces through which this can be 
performed: 

1. Simulation of a single execution of the model. This can be thought of 
a debugging engine, in which we can either automatically simulate a 
certain number of steps (using a random number generator for proba- 
bilistic choices), or manually select the next state. The interface also 
supports backtracking, and allows us to explore the state space of the 
model. 



Approximate probabilistic model checking 54 . Given a quantitative 
property (of the form P =? or R =?), the model is simulated multiple 
times, and an approximate value for the property is given by the ex- 
pected value of the property over all runs. We can specify the desired 
precision — i.e. that the actual value lies within e of the approximate 
value with a given confidence interval — and PRISM will compute the 
required number of samples to ensure this. 



7.1.4 Experiments in PRISM 

One additional feature provided by PRISM is its support for so-called exper- 
iments. The idea is that we often want to verify not just one property, but 
a class of properties where one parameter is varied. For example, we might 
want to know the probability of a path formula holding as we change the 
value of some variable, or as we change the time bound of a path operator. 

To perform experiments, we simply declare some constants in the PRISM 
model, but leave them undefined. When we come to specify a property, we 
can then use these variables, and PRISM will prompt us for the values to 
use. This is typically specified as a start and end value, and a step size, 
indicating a range of values. PRISM will then model check the property for 
each value of the variables. 

Note that the experiments feature can make use of either the stochastic 
model checker or the simulator. In both cases, we perform a parameter sweep 
such that we either verify the same property for a set of different models, or 
a set of different properties for the same model. The intent is often to plot 
a graph of the probability or reward value of a quantitative property with 
respect to the parameter we vary. Note that experimentation and simulation 
are two different and complimentary features. 
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7.2 MRMC (version 1.4.1) 



The Markov Reward Model Checker (MRMC) |72 is an explicit state stochas- 
tic model checker, whose focus has historically been on reward-structured 
Markovian models. It supports five different modelling formalisms: DTMCs, 
CTMCs, discrete time Markov reward models (DMRMs), continuous time 
Markov reward models (CMRMs), and continuous time Markov decision 
processes (CTMDPs). Technically, MRMC supports CTMDPIs, which are 
CTMDPs that contain internal non-determinism in addition to external non- 
determinism that is controlled by an environment. 

Unlike PRISM, the MRMC tool does not accept a modelling language 
directly — instead, it takes an explicit description of the state space, transi- 
tion system, state labels, and reward structures of the model as input. This 
is given by number of simple text files that explicitly list every state in the 
model, and every transition between states, etc. As a command line tool, 
it is designed to be used a back-end to other systems, in which higher level 
compositional modelling languages are used — for example, PEPA and the 
PRISM language. 
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Figure 7.3: Overview of the structure of MRMC 



A diagram of the structure of MRMC, taken from [H], is shown in Fig- 
ure 7.3 There are two fundamental model checking engines: 
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• Explicit- state stochastic model checking — this uses the model checking 
algorithms that we mentioned briefly in the previous chapter. Unlike 
PRISM, there is no use of symbolic model checking, but bisimulation 
minimisation techniques are used to reduce the size of the state space. 

• Statistical model checking — this uses discrete event simulation to vali- 
date a property, returning confidence bounds on its correctness. There 
is also a hybrid mode for steady state properties, in which the probabil- 
ities of reaching bottom strongly-connected components are computed 
numerically, but the steady state behaviour is explored by simulation. 

MRMC supports the following logics for each modelling formalism: 





PCTL 


PRCTL 


CSL 


CSRL 


DTMC 


/ 








DMRM 




/ 






CTMC 






/ 




CMRM 








/ 


CTMDPI 






/t 





^For CTMDPIs, only time-bounded reachability properties are supported. 

We will not describe the syntax for property specification here, as it is 
essentially the same as that of the logics described in the previous chapter. 

7.3 Other Model Checking Tools 

In this section, we present a brief description of other stochastic model check- 
ing tools. 

7.3.1 RAPTURE (version 2.0.0) 

The RAPTURE tool [8, 69] provides support for verifying quantified reach- 
ability properties over a Probabilistic Transition System (PTS). A PTS is 
similar to an MDP, in that given a state and an action label, there is a prob- 
abilistic choice as to the next state to transition to. Unlike an MDP, however, 
it is reactive, and two PTSs can perform a CSP-style synchronisation over 
shared actions. In this sense, it is more similar to the process algebra IMC, 
described in Section 15. 21 

In RAPTURE, a system is specified in terms of a number of channels 
and processes, as well as a set of a global initial states and a set of global 
final states. The latter describes the set of states that we are interested in 
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reaching, and so can be thought of as specifying the reachability property as 
part of the system. In addition to calculating such reachability probabilities, 
RAPTURE employs a number of reduction techniques in order to mitigate 
the problem of state space explosion. 



7.3.2 PASS 

The probabilistic model checker PASS (Predicate Abstraction for Stochastic 
Systems) can be used to analyse concurrent probabilistic programs which 
map to DTMCs or MDPs with infinite states. It is based on predicate ab- 
straction and automatic abstraction refinement. Models are specified in a 
variant of the PRISM language, which allows infinite data types. 

Since PASS does not unfold the entire state space of the original model, 



the tool is not restricted to finite models — unlike, for example, Rapture 35 
and the magnifying-lens abstraction of [9]. Instead, PASS takes the ap- 
proach of counterexample-guided abstraction refinement (CEGAR), which 
uses predicate abstraction to maintain a finite abstract model. Analysis of 
this abstract model is typically very efficient since it has a small state space. 
Importantly, the analysis is safe, in that it yields probability intervals that 
are guaranteed to contain the probabilities of the corresponding properties 
in the original model. The size of the interval is used to quantify the ap- 
proximation error caused by the abstraction. If this error is too large, the 
abstraction is refined — using diagnostic information (effectively, a coun- 
terexample) obtained from the abstract model. 



This process is described in 56,96 . Note that a major difference to 



conventional CEGAR for predicate abstraction is that the counterexamples 
are Markov chains, rather than single paths. The tool makes use of SMT 
solvers for computing finite abstractions, numerical methods for computing 
probabilities on these abstractions, and interpolation as part of the abstrac- 
tion refinement mechanism. PASS has been successfully applied to network 
protocols, and serves as a test platform for different refinement methods. 



7.3.3 PARAM 

PARAM is a tool for handling parametric variants of models specified in a 
variant of the PRISM language. It extends the PRISM language with the 
possibility of defining unknown parameters, and later using such parame- 
ters to specify probability distributions. PARAM is capable of computing 
unbounded reachability probabilities for parametric DTMCs. 

To solve this problem, Daws [36] devised a language-theoretic approach, 
in which the transition probabilities are taken to be letters in an alphabet. In 
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this way, the model can be viewed as a finite state automaton. Given this, a 
regular expression describing the language of the automaton is computed, 



using the state elimination 66 method. The regular expression is then 
recursively evaluated, which results in a rational function over the parameters 
of the model. Gruber and Johannsen [50] have shown, however, that the size 
of the regular expression of a finite automaton explodes — for an automaton 
with n states, it has size n e( - logn \ 

The core of PARAM is also rooted in the state elimination algorithm. 



The key difference to 36 is that instead of post-processing a (possibly pro- 
hibitively large) regular expression, the state elimination and rational func- 
tion computation stages are intertwined. More precisely, regular expressions 
are not used in the state elimination step — instead, the edges are labelled 
directly with the appropriate rational function to represent the flow of prob- 
abilities. This also means that the process remains in the domain of Markov 
chains, rather than working on a finite automaton representation. 

In addition to DTMCs, PARAM can also handle special classes of MDPs, 
as well as reachability rewards for Markov reward models. To speed up 
computations, it computes the (strong and weak) bisimulation quotient of 
the parametric model. 



7.3.4 INFAMY 

INFAMY is a tool for model checking CSL formulae on infinite state CTMCs, 
which are specified in a variant of the PRISM language. It implements the 
first CSL model checking algorithm to use truncation. This algorithm enables 



the automatic analysis of infinite (or very large) CTMCs, and, unlike 90,91 
it can be applied to arbitrarily structured (finite or infinite) CTMC models. 
Given a CSL property, the algorithm proceeds in two phases: 

1. A finite truncation depth is computed, which is sufficient to check the 
property up to a given accuracy. 

2. The property is verified on the resulting truncated model, using the 



CSL model checking algorithm for finite CTMCs 17,74 



Note that computing a sufficient truncation depth for CSL model checking 
is more challenging than for transient analysis, since the required depth de- 
pends not only on the desired precision and the characteristics of the model — 
formulae involving until operators and nested sub-formulae must also be han- 
dled. Nevertheless, results for transient analysis form an important building 
block of the algorithm. 
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INFAMY provides a number of methods for finding a stopping criterion 
for the state-space exploration. Currently, the supported methods are Uni- 



form, Layered, FSP, and FSP exp — for a comparison, see 51,52 . It is 
interesting to observe the tradeoff between the time at which the state space 
exploration is stopped, and the memory needed to store the finite truncation 
of the state space. In addition to CSL properties, INFAMY can also verify 
certain reward properties. 



Part V 
Performance Evaluation 
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Chapter 8 



Tools for Performance 
Evaluation 



In this chapter, we will give an overview of the tool support for the per- 
formance evaluation techniques that we described in the previous chapter. 
Since there is a certain degree of overlap between performance evaluation 
and stochastic model checking, there are many tools that offer similar func- 
tionality to one another. Our focus in this chapter, however, will be on the 
tool support for analysing performance properties that are not expressed in 
a logic. 

Of the many performance evaluation tools that are available, we will focus 
our attention on three in particular: 



PEPA Tools (Section 8.1[ ). We introduced the Performance Evaluation 



Process Algebra (PEPA) in Section 5.1, and we will give, in this chap- 



ter, a survey of the main tools that are available for analysing PEPA 
models. 



Mobius (Section 8.2). This is a popular tool that supports multiple 
high-level modelling formalisms, which can be composed to form hier- 
archical models. There are a number of analysis engines, based around 
numerical solution of Markov chains, and discrete event simulation. 



MATLAB (Section 8.3). Many mathematicians write specific models 
directly in the MATLAB programming language, which provides in- 
built support for matrices and standard mathematical operations. 
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8.1 PEPA Tools 

There has been a long history of tool support for PEPA, initially based 
around numerically computing the steady state distribution of the under- 
lying CTMC, but subsequently encompassing first passage time analysis, 
stochastic simulation, fluid-flow approximation and model checking of Con- 
tinuous Stochastic Logic (CSL). The original PEPA tool was the PEPA Work- 



bench 46 , which was developed in 1994 as two independent versions — one 
written in SML, and one in Java. Over time, the Java tool gained dominance, 
with a number of researchers and masters students extending its function- 
ality. However, due to the involvement of many people, and the lack of a 
structured design, it eventually became bloated and too difficult to maintain. 



In 2006, the PEPA plug-in project 94 began, as an attempt to re- 
implement the PEPA workbench using good design principles. Rather than 
a stand-alone application, it was decided to develop a plug-in for the Eclipse 
platform [Tj, both to make use of a standard interface, and to allow eas- 
ier integration with other tools. In addition to numerical solution of mod- 



els, the PEPA plug-in supports fluid-flow approximation 60 and stochastic 



simulation 23 of PEPA models. More recently, support for compositional 



abstraction and CSL model checking has been added 92 . 

In parallel to the PEPA workbench and the PEPA plug-in, tools have 
been developed for transient analysis of PEPA models. In particular, the 
Imperial PEPA Compiler is a tool that compiles a PEPA model into the input 
language of HYDRA [24], allowing distributed computation of response time 
distributions. Stochastic probes |3l] are used to specify the response time 
to measure. Recently, the Imperial PEPA Compiler has been superseded by 
the International PEPA Compiler (IPC) |2|, which can be used both as a 
stand-alone application and through Eclipse — allowing integration with the 
PEPA plug-in. 

In addition to these language-specific tools, PEPA is also supported by 
a number of more general purpose performance evaluation and stochastic 
model checking tools. A subset of PEPA (allowing only active-passive syn- 
chronisation) is directly supported by the PRISM model checker (63) (see 



Section 7.1[ ), and Mobius [34] supports an extension of PEPA, called PEPA 



which we will describe in Section 18.21 

A summary of the four main tools for PEPA, and their functionality, is 



shown in Table |8.1| Note that we have not labelled PRISM as supporting 
stochastic simulation, because it does not support simulation-based perfor- 
mance evaluation in the same way as the PEPA plug-in and Mobius do. 
Currently, PRISM only supports discrete event simulation in the context of 
debugging and statistical model checking. 
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PEPA Plug-in 


IPC 


PRISM 


Mobius 


Steady State Solution 


/ 


/ 


/ 


/ 


Stochastic Probes 




/ 






CSL Model Checking 


/ 




/ 




Stochastic Simulation 


/ 






/ 


Fluid-Flow Approximation 


/ 









Table 8.1: Performance evaluation features of the main PEPA tools 



We will discuss stochastic probes in more detail in Section |8.1.1[ and 
population-level analysis using stochastic simulation and fluid-flow approxi- 
mation in Section |8. 1.21 



8.1.1 Transient Analysis 

The International PEPA compiler (IPC) [2] is a tool for computing passage 
time distributions for PEPA models. It supports an extension of PEPA 
that allows immediate actions (i.e. actions that are instantaneously taken), 
functional rates [61] , and arrays of processes (which all cooperate over the 
same set of action types). Note that transitions corresponding to immediate 
actions are always given priority over stochastic transition^] 

To illustrate the idea of a passage time query, let us consider a simple 
PEPA sequential component. This corresponds to a client, sending a request 
to a server and waiting for a response: 



Client 
Client' 



def 



(request, r). Client' 
= (response, T). Client 



This could potentially be composed with a complex model of a server, which 
processes the request in multiple stages, or has to contend for some shared 
resource such as a remote database. From the point of view of the client, 
however, we might like to ask a simple question — "after I make a request, 
how long do I have to wait before I receive a response?" 

To ask such queries, IPC supports a language called extended Stochastic 
Probe (XSP) (3l]. The basic idea is to specify when we start measuring the 
passage time, and when we stop measuring it. In the case of our example 
query, we start when we observe a request activity, and we stop when we 



lr This corresponds to the maximal progress assumption in Interval Markov Chains 
(IMC). See Section [5^21 
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observe a response activity. The stochastic probe would therefore be as 
follows: 



The meaning of Client :: P is to attach the probe P to the Client component. 
This is important if we want to measure the response time of an individual 
client — there may be other components in the model that also perform 
request and response activities with the server, but we are only interested in 
those activities that involve the client. 

In general, we might want to ask much more complex queries than the 
above, and so it is not sufficient to identify a single pair of activities that 
cause us to start and stop measuring the passage time. Instead, we are able 
to specify regular expressions of activitie^J An example of such a stochastic 
probe is as follows: 

Client :: (request-^ \ request^) : start, [response A , response B ) : stop 

This specifies the passage time between observing either a request^ or a 
request^ activity, and observing a response A activity followed by a response B 
activity. 

Given a PEPA model and a stochastic probe, IPC compiles the probe 
into a PEPA component that records the state of the probe (i.e. whether 
it is measuring or not measuring the passage time), but does not affect the 
state of the model. This is composed with the original PEPA model, and 
may cause its state space to increase — depending on the complexity of the 
regular expressions in the probe. The approach is similar to the automata- 
theoretic approach to LTL model checking, which we discussed briefly in 
Section 16.1.21 

At the backend of IPC is the ipclib/HYDRA toolchain (24| . HYDRA, 
the HYpergraph-based Distributed Response time Analyser |38| , is a tool for 
computing first passage time distributions of a CTMC, using uniformisatiorj^J 
It does so in a distributed fashion, using a hypergraph partitioning of the 
state space — each processor is assigned a set of states in the CTMC, in such 
a way that the required communication between processors is minimised. 



2 XSP also allows us to use state specification as guards — that is to say, we only 
observe an activity if the current state of the model satisfies a certain property. We will 
not describe this here, but refer the reader to [31). 



3 HYDRA is an extension of DNAmaca, which is a tool for solving the steady state 
distribution of large Markov chains. 



Client :: request : start, response : stop 
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8.1.2 Population-Level Analysis 

In PEPA, we often want to write a model that consists of a certain number of 
identical components in parallel. For example, if we want to model a system 
where four clients connect to two servers, we could write a system equation 
that looks something like the following: 

(Client || Client || Client \\ Client) w (Server \\ Server) 

{ request, response } 

To make it easier to specify such models, an aggregation combinator was 
introduced into the language. If we want to represent n copies of a sequential 
process P in parallel (with no shared activities between them), we can simply 
write P[n]. This means that the above system equation can be simplified to: 

Client[A] w Server[2] 

{request, response} 

The problem with models of this form is that the underlying state space can 
become intractably large even for relatively small numbers of components. 
Let us consider a term P[n], where the sequential process P has m different 
configurations (|ds(P)| = m). If we naively generate the state space of the 
underlying CTMC, we will find that it contains m n states. In other words, the 
state space grows exponentially large with respect to number of components. 

We can be cleverer than this if we notice that two components in the 
same state are indistinguishable from one another — more precisely, they are 
strongly bisimilar. For example, we cannot distinguish between Pi || P2 \\ Pi 
and Pi || Pi || P2, since observably they both correspond to two processes in 
state Pi and one process in state P 2 . We can therefore aggregate such states 
in the CTMC induced by the PEPA model, since they exhibit (ordinary) 
lumpability. In general, the number of states in the aggregated process P[n] 
willbaS 

/ n + m — l\_(n + m— 1)! 
\ n J n\(m-l)\ 

Whilst this is a significant improvement, the number of states still grows 
quickly with n — for a fixed m, the number of states will grow as a polynomial 
in n of degree m — 1. 

To combat this, the PEPA plug-in supports two technique^} 

4 Combinatorically, this is the number of ways we can place n indistinguishable balls 
into m distinguishable urns. 

5 Initially, these analyses were performed by compiling PEPA into the input language 
for Dizzy [88] , a chemical kinetics simulation package. They are now, however, supported 
natively by the PEPA plug-in. 
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1. Stochastic simulation [23]. We simulate the CTMC induced by the 
PEPA model using Gillespie's algorithm |45] (and more recent im- 
provements on it). The essential idea is a discrete-event simulation 
of a reaction-based representation of the CTMC. 

2. Fluid-flow approximation [60]. Rather than having a discrete state 
space that describes the number of components in each state, we ap- 
proximate this by a continuous state space. This leads to an alternative 
semantics for PEPA, which describes the model as a system of ordinary 
differential equations (ODEs). In particular, there is one differential 
equation for each state of a sequential component, but the number of 
equations does not grow with the number of components. 

Both of these approaches take a population-level view of the PEPA model. 
In other words, we use PEPA to model the behaviour of the system at the 
level of an individual component, or species, and then use this to analyse 
the emergent behaviour of the population. In doing so, we lose information 
about an individual — for example, we can no longer ask how long it takes 
for an individual client to receive a response from a server, but we can ask 
questions about how the number of clients waiting for a response changes 
over time. 



8.2 Mobius (version 2.3) 



Mobius 34 is a multi-paradigm performance evaluation tool. It supports 
multiple modelling languages, which can be used to model individual com- 
ponents of a system, and then be combined using common notions of com- 
position. A model in Mobius essentially consists of a number of states, along 
with various actions that allow it to change state. Actions have a dura- 
tion, but unlike in most stochastic process algebras, they can be generally 
distributed^ 

There are two main types of analysis in Mobius, and the analysis engines 
are common to all the modelling formalisms: 

1. Distributed discrete- event simulation — this can be used to analyse 
both the transient and steady state behaviour of a model. It supports 
distribution of simulation runs to multiple machines over the network, 
and automatically collects and collates the results (this is done by au- 
tomatic remote login to other machines via rsh or ssh). 



6 M6bius has in-built support for the binomial, deterministic, gamma, exponential, Er- 
lang, beta, hyper-exponential, negative binomial, geometric, uniform, triangular, Weibull, 
conditional Weibull, normal, and log-normal distributions. 
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2. Numerical solution — if a model contains only actions with exponen- 
tially distributed durations, then it can be solved numerically — both 
for transient and steady state analysis. If a model contains both expo- 
nential and deterministic transitions, and only one deterministic tran- 
sition is enabled at any time (whose duration does not depend on the 
state of the model), then its steady state can be numerically solved. 
Numerical solution, however, requires the model to be small enough to 
fit in memory. 

Models in Mobius are inherently hierarchical — individual components 
of the system are modelled separately, and then composed to build a new 
model. This can itself be a sub-model of a larger composed model. Com- 
position is primarily based around having shared state between sub-models, 
although actions can alternatively be shared. There are three ways of speci- 
fying composition in Mobius: 

1. Replicate/join — the composition is described by a tree, whose leaf 
nodes are individual components. A parent node is either a replicate-k 
node, which has a single child and corresponds to instantiating k copies 
of that child, or a join node, which has multiple children and composes 
them. Both replicate and join nodes must specify which variables are 
shared between their children, and which are local. 

2. Graph composition — this is an alternative to replicate/join, where 
there is no requirement that the graph forms a tree, and there is no 
replicate node. This allows us to specify in a more intuitive way the 
composition of more than two sub-models that share certain elements 
of one another's state. Note however that any graph composition can 
be expressed clS cL replicate/join, and vice versa. 

3. Action synchronisation — rather than sharing state between sub-models, 
we can share their actions. The enabling conditions of a shared action 
are the union of its enabling conditions in each of the sub-models, and 
the new rate of the action can be specified by the user. Like repli- 
cate/join, an action synchronisation is described as a tree. 

There are four main modelling formalisms supported by Mobius — stochastic 
activity networks, buckets and balls, PEPA, and fault trees. These are de- 
scribed in detail in the Mobius user manual |2), but we will give an overview 
here for completeness. 
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8.2.1 Stochastic Activity Networks (SANs) 

Stochastic Activity Networks (SANsJ^j |82;j are similar to stochastic Petri nets 
(SPNs) |8i] , in that they are a graphical formalism. A SAN consists of four 
primitive elements: 

• A place corresponds to a state of the model, and can contain a num- 
ber of tokens. The number of tokens assigned to a place is called its 
marking. 

• An activity is an action, linking a set of input places to a set of output 
places. Activities can be either timed or instantaneous. 

• An input gate is a Boolean function on the markings of the input places 
of an activity, and must evaluate to true in order for the activity to be 
enabled. 

• An output gate is a function that defines the marking changes that 
occur in an output place when an activity fires. 

8.2.2 Buckets and Balls 

Like SANs, buckets and balls are a graphical formalism, but are much sim- 
pler. These are used to model components where the capabilities of an SAN 
are not needed (e.g. the use of input and output gates). There are two 
primitive elements: 

• A bucket corresponds to a state in the model, and can contain a number 
of balls. 

• An transition corresponds to an event that transfers balls between two 
buckets, and is drawn as a directed edge. The cardinality of a transition 
determines how many balls will be transferred, and it can only fire if 
there are sufficient balls in the source bucket. As with SAN activities, 
transitions can be either timed or instantaneous. 



8.2.3 PEPAi 



PEPAfc is an extension of the PEPA language (described in Section 5.1), 
which adds the following language features: 



7 Note that these should not be confused with Stochastic Automata Networks 85 , 
which have the same acronym, but are very different. 
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• Sequential process definitions can be given parameters, which corre- 
spond to natural numbers. 

• Activities can be prefixed by guards, which are Boolean predicates on 
the parameters. An activity 

An example of a PEPA^ component is the following, where P[a] is defined 
for all a > 0: 

P[a] = [ a > 0] (a,r).P[a - 1] 
+ [ a = 0] (a,r).P[a} 

Note that we can always translate a PEPA& model into PEPA, by expanding 
out the sequential process definitions, starting from the system equation. 
There is no syntactic guarantee that the model will be finite, however. 



8.2.4 Fault Trees 



Fault trees 39 are a formalism used to analyse the reliability of a system, by 
analysing the relationship between the failure of individual components, and 
of the entire system. The basic idea is that a system failure is described as a 
logical combination of component failures — Mobius also supports a subset 
of dynamic fault trees, allowing failures to also depend on the sequence in 
which components fail. 

The elements of a fault tree can be either active, or inactive. There are 
three types of element: 

• A node is a failure state of the system. The root element of a fault tree 
must always be a node, but nodes can also be present at intermediate 
levels of the tree. 

• An event is a failure of a component in the system. Events form the 
leaf elements of the fault tree, and correspond to activities in the other 
Mobius formalisms. 

• A logic gate connects a number of elements of the fault tree (its in- 
puts) to a parent element (its output), and is triggered by a condition. 
Logic gates may be static (AND, OR, XOR, and if-of-N^]) , or dynamic 
(priority AND). The output state of a static gate depends only on the 
current state of its inputs, whereas that of a dynamic gate may also 
depend on the past state of its inputs. Mobius only supports one dy- 
namic gate — the priority AND, whose output becomes active if and 
only if its inputs become active in a certain order, specified by the user. 



3 The output of a if-of-N gate is active if and only if K of its inputs are active. 
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8.3 MATLAB 

Rather than using tools that support language-based or graphical formalisms, 
mathematicians that build stochastic models tend to work at a lower level, 
using parameterisation to compactly specify a model. To this end, MAT- 
LAB 13] is a widely used programming language, since it natively supports 
vectors and matrices, and various standard mathematical operations. Whilst 
this lacks the compositionality of higher level language-based formalisms, it 
is in many ways closer to the parametric specification of models preferred by 
mathematicians. 
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